What is the Illuminate Integration Gateway?

The Illuminate Integration Gateway, also called the Dataport, is a tool used to provide the Illuminate platform with access to data that is located in physically and logically secure configurations, such as behind a firewall in the organization’s data center. The Integration Gateway provides a secure, encrypted channel to transport this data to the Illuminate cloud environment, where Illuminate creates a secure Application Program Interface (API) access platform.


How it works

N2N provides a Docker container that runs Illuminate's Integration Gateway (IIG). This is installed on a Virtual Machine (VM) that is provided by the University and typically hosted on the University's network. The IIG is used to facilitate secure communication between the University database and the Illuminate Platform without having to allow direct external communication to the actual database. VM setup is operating system specific, please refer to your onsite System Administrator for VM setup.

What is the current version of the Integration Gateway?

Illuminate Integration Gateway is at Version 4.3

*Note: Prior versions of the Illuminate Integration Gateway were branded "Illuminate Dataport"

What do we provide?

N2N provides a Docker image with the following software packages included:

  • Tomcat 9.0.20
  • JVM 11.0
  • Illuminate Integration Gateway Project 4.3 (the N2N application code)

General System Requirements

General system requirements for installing the Integration Gateway:

Software

  • Docker 17.x version or higher

Hardware (VM)

  • 4 to 8 CPU Cores (processors)

  • 8GB to 16GB RAM

  • Enough disk space to run the OS and other supporting software (recommended disk space 40+ GB)


  • Typically an 8 GB Ram server with 4 Core Processor can serve about 700-800 transactions per second.

  • For more information about Docker, prerequisites visit this Docker page

What is Docker and why does N2N use this?

What is Docker?

Docker is the world's leading software containerization platform. Docker containers wrap a piece of software in a complete file system that contains everything needed to run: code, run time, system tools, system libraries – anything that can be installed on a server. This guarantees that the software will always run the same, regardless of its environment. Containers isolate applications from one another and the underlying infrastructure while providing an added layer of protection for the application.

For more details about Docker please visit this Docker page

What is Docker Compose?

Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration.

For more details about docker-compose please visit the Docker Compose page

Why does N2N use Docker for the Illuminate Integration Gateway?

The Docker platform provides N2N with a simple, effective way to deliver the right package of software to the Illuminate Integration Gateway. Docker allows N2N to combine tested versions of software with validated configuration files into one deployment package. This process greatly reduces the number of manual steps, which improves reliability and robustness, as well as making deployment faster!

Docker Installation Instructions

Docker CE (Community Edition) is available on multiple platforms. Use the following matrix to choose a supported operating system that provides the best installation path for you. Click on the platform to navigate for detailed documentation on the docker site. We provide detailed instructions for installing Docker and the Integration Gateway on CentOS along with other required configurations. Other Linux implementations will be similar.

Supported Platforms

Platform

Docket CE x86_64

CentOSYes (Recommended)

Ubuntu

Yes

Debian

Yes

Red Hat Enterprise Linux

Yes

Fedora

Yes

Microsoft Windows 10

Yes

macOS

Yes

Microsoft Azure

Yes

Amazon Web Services

Yes

Oracle Linux

Yes

SUSE Linux Enterprise Server

No

Microsoft Windows Server 2016

No


For more details please visit docker supported-platform documentation

Detailed instructions for installing Docker & Docker Compose (on CentOS)

Prerequisites

  • An SSL certificate ready (self-signed is not supported) which includes .crt file (the .crt file must have the public, root, and intermediate certs, with public cert on top of the file) and a private key. (See SSL Certs FAQs at the end of this document)

  • Client will need to assign a DNS to their public facing IP of the server which will host the dataport. That DNS should match the SSL cert assigned to the dataport. 

  • Ports 443, 80, 2376 (2376 port is optional) need to be open on the VM instance where Docker and Integration Gateway are installed

  • Get the authorization credentials from N2N to pull the Docker image

Docker Installation

There are two ways to install Docker Engine

  1. Install using the yum package manager (Recommended)
  2. Install using the instruction from https://docs.docker.com/install/linux/docker-ce/centos/

Install with yum

  • Log into your VM as a user with sudo rights to install and manage services or root user. If you are logged in as root you do not need to add the "sudo" at the beginning of each command.
  • Make sure your existing packages are up-to-date using yum
sudo yum update -y
  • Add the yum repository if it doesn’t exist on your server where you are installing Docker. 
  • We will also need to enable the Redhat "extras" repository for RHEL 7 (The third command to enable the Redhat extras repository will only need to be run if you are running RHEL 7).
sudo yum install -y yum-utils

sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

sudo yum-config-manager --enable rhel-7-server-rhui-extras-rpms
  • Install the Docker package for RHEL 8.  

sudo yum install -y --nobest docker-ce docker-ce-cli containerd.io
  • Install the Docker package for RHEL 7.  

sudo yum install -y docker-ce docker-ce-cli containerd.io
  • Enable the Docker service.  This is needed so docker will start whenever the system is restarted

sudo systemctl enable docker.service
  • Start the Docker daemon

sudo systemctl start docker
  • Verify Docker has been installed correctly by running a test image in a container

sudo docker ps
  • The screenshot below will show the installation is done properly.

Now that Docker installation is complete, the next step is to install Docker compose. 

Docker Compose Installation

  • Run this command to download the current stable release of Docker Compose

sudo curl -L "https://github.com/docker/compose/releases/download/1.24.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose


If you have problems installing with curl, see Alternate Installation Options


  • Apply executable permissions to the binary. 

sudo chmod +x /usr/local/bin/docker-compose


If the command docker-compose fails after installation, check your path. You can also create a symbolic link to /usr/bin or any other directory in your path

For Example:

sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

For more details about Docker Compose, please visit here


  • Test the docker-compose Installation

docker-compose --version

O/P: docker-compose version 1.24.0, build 1110ad01 it will show docker-compose version


Integration Gateway
Installation

There are two ways to install Integration Gateway

  1. Install using the YAML script (Recommended, & detailed in this document)
  2. Install using manual steps (not included in this document)

Authorization to Access Integration Gateway Docker Images

  • Use the credentials provided by N2N to access Integration Gateway docker images
The command/credentials will look like:
docker login -u="n2ndevops+User" -p="xxxxxxxxxxxxxxxxxxx" quay.io

Integration Gateway Installation using YAML Script

  • Login as root on to the VM

  • Create a folder with your preferred name

For Example:
mkdir n2ndataport
  • Download the docker-compose.yml file which attached to the bottom of this document.

  • Place this docker-compose.yml file in the folder 'n2ndataport' which was created in the above step.
  • Run the docker-compose.yml file to deploy both Nginx and Integration Gateway Docker containers. Use the below command;
docker-compose up -d


Once docker-compose up -d is executed:

  • Default Docker network is created with the folder name where the docker-compose.yml file is located (in the screenshot above n2ndataport_default docker network is created based on folder n2ndataport)

  • Then in the next step, it started pulling the Integration Gateway image and Nginx image

  • In the final step, Integration Gateway and Nginx containers are created

The output after running the above command will look like the screenshot below

  • Run docker ps command to view the installed/running containers.

          In the screenshot below, we can see that the Integration Gateway (Dataport) and Nginx containers are up and running.

Applying the SSL Certificate

  • Now copy all your SSL certificate files (such as the .crt and .key files*) to the “certs” folder inside the Nginx container
docker cp <your file name>.crt nginx:/certs/<your file name>.crt 
docker cp <your file name>.key nginx:/certs/<your file name>.key     


File extensions .crt and .key are used as an example, use appropriate valid extensions as needed.

  • FAQ: What type of certificates are we using for connecting to your Integration Gateway?

          We are using X.509 type certificates.

  • FAQ: Should SSL certificates point to the server or docker containers?

          Here SSL certificates should be in relation with host_name or domain name (while generating certificates).

          These certificates should point to the docker containers. Here, in this case, we are placing certificates in the nginx container.

  • Login to the Nginx container

docker exec -it nginx bash
  • Open default.conf file:

cd /etc/nginx/conf.d/
vim default.conf



  • Add "ssl" at line 2 as shown below

    • Example: listen 443 ssl;

  • Change server_name to yourdnsname.com at line 3 and 22

  • Uncomment lines 4 and 5 (Note: Do this step after copying certificates to nginx container)

  • wq! (save the configurations)
  • Now test Nginx config using this command (the command checks whether the certificates are valid)

nginx -t
  • If the validation is successful, exit from the nginx container. Otherwise, review the error and return to the Nginx default.conf file

exit
  • Restart the Nginx container

docker restart nginx

Test connection

  • Test the connection using the curl command below. It should return a response: "Not a valid request.”. Note that you need to configure the command below to match your organizational nomenclature.

curl -X GET https://{server_name(or)host_name}/idp

Optional step to autostart the containers after a server reboot

If a server is rebooted, restarted or start/restart, the docker engine must be restarted. To accomplish that automatically follow the steps below

1. chkconfig docker on

2(i). docker update --restart=unless-stopped $(docker ps -q)
Note: This command starts all the containers within the docker engine of the server
(OR)
(ii) docker update --restart=unless-stopped containerID/ContainerName
Note: This command will start only the specified container

Establishing a connection between Illuminate, Integration Gateway and the SIS Database

Note: Firewall rules need to set up by this time. Whitelist N2N NAT IPs below

QA: 193.122.133.255, and 23.20.165.255

PROD: 129.213.17.154, 129.213.178.3, and 54.204.165.208

Step 1 - Adding Integration Gateway URL in Illuminate app

Note: In this step, we will configure the Integration Gateway details in the Illuminate App.

Note: If you do not have an Illuminate account please contact appsupport@n2nservices.com 

  • Configure the Integration Gateway details within the Illuminate App's Connection Module by following below steps
    • From the left-side menu, select the connection button.
    • Next, select Setup Dataport from the list of options
    • Click on GET STARTED
    • Enter the 'Dataport Name' field and the 'Dataport URL' field then click on the publish button on the screen.
      • Note: Dataport Url example looks like https://{server_name(or)host_name}
  • Note: You can also follow the steps from this link Step by Step visual instructions for Step1
Step 2 - Add the database provider in the Illuminate app

Note: This step is also performed in the Illuminate App.

  • Configure the Database provider details within the Illuminate App's Connection Module using the following steps:
    • From the left-side menu, select Connections.
    • Click the ADD PROVIDER button above the provider list index table
    • Next, under Provider Details select the Provider Type as Database.
    • Select the appropriate database under Provider (i.e Oracle or SQL Server per your SIS).
    • Enter the desired name for Provider Name.
    • Enter a Connection Name for this setup.
    • Enter a description for this connection.
    • Click Save.
    • A popup will appear with authorization details that should be copied and used to run the database curl command in Step 3. 
      • Note: The generated token will expire in 5 mins.
  • Note: You can also follow the steps from this link Step by Step visual instructions for Step 2
Step 3 - Connecting the Integration Gateway to the Database(s)

Note: The instructions below are to be performed on the Integration Gateway Server.

In this step, we will establish the connection between the Integration Gateway and the database by creating a property file.

Note: Within these instructions, EMPDB.properties is used as an example. EMPDB can be replaced with an appropriate database name. The database properties are stored in an encrypted format in EMPDB.properties.

Use the below method to configure the EMPDB.properties:

The properties are generated using a curl command. For security, please turn off Linux history before running the curl command, using the following set command:
set +o history
  • The Illuminate Integration Gateway includes a web service that will create these properties in the system.

Step 4 - Creating the request body with the database properties

This service needs a JSON request body as shown below, edit this sample to include your specific database server information for Oracle or SQL Server:

{
    "DBHost": "Server hostname or IP", // database host
    "port": "1234", // database port
    "SID": "", // sid or service name for oracle providers
    "ServiceName": "servicename",
    "databaseName": "", // databaseName is for SQL server providers
    "DBUsername": "name", // DB username or schema Name
    "DBPassword": "***********" // DB password
}


  • A value should be populated for either SID or ServiceName but not both.
  • Ensure there are no carriage return/line feed (CR/LF) characters in the parameters before running the full curl command.


{ "DBHost": "0.0.0.0", "port": "1234", "SID": "", "ServiceName": "servicename", "DBUsername": "username", "DBPassword": "***********" }


{ "DBHost": "0.0.0.0", "port": "1234", "databaseName": "", "DBUsername": "username", "DBPassword": "***********" }


{ "DBHost": "0.0.0.0", "port": "1234", "databaseName": "", "DBUsername": "username", "DBPassword": "***********","informixServer": "" }

Step 5 - Create & run the CURL command
  • Run the following command to retrieve the Docker container IP (in this case we need the Integration Gateway container port)
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' dataport
  • Replace "dataport_container_ip:port" with the response body from the last step, in the following curl.  invoke the below service (Recommended)
curl -X POST 'http://{dataport_container_ip:port}/idp/addDBproperties' -H 'Authorization:Token from Illuminate connections page in Step 2' -H 'Content-Type: application/json' -d '{"DBHost": "0.0.0.0","port": "1234","SID": "","ServiceName": "servicename","DBUsername": "username","DBPassword": "***********"}'

      (OR)

  • You can also use DNS (which is assigned to the Integration Gateway) to add DB properties file
curl -X POST 'https://{your_dns_name}/idp/addDBproperties' -H 'Authorization:Token from Illuminate connections page in Step 2' -H 'Content-Type: application/json' -d '{"DBHost": "0.0.0.0","port": "1234","SID": "","ServiceName": "servicename","DBUsername": "username","DBPassword": "***********"}'


If the Integration Gateway is on a load-balanced network and you use the DNS method to add the properties it will not update in all the containers. In this case, use the container IP to individually update the properties.


Load Balancing the Integration Gateway

Note: Recommended but optional

Load balancing refers to efficiently distributing incoming network traffic across a group of servers. A Load Balancer acts as the "traffic cop" sitting in front of your servers and routing requests across all servers capable of fulfilling those requests in a manner that maximizes speed and capacity utilization and ensures that no one server is overworked, which could degrade performance.If a single server goes down, the load balancer redirects traffic to the remaining online servers. When a new server is added to the server group, the load balancer automatically starts to send requests to it. 


If the Integration Gateway is on a load-balanced network, please ensure appropriate steps are followed to update the database properties in all relevant servers/containers. 


For general background information, see this article about Load Balancing

Sample load balancing diagram leveraging Docker