What is the Illuminate Integration Gateway?

The Illuminate Integration Gateway, also called the Dataport, is a tool used to provide the Illuminate platform with access to data that is located in physically and logically secure configurations, such as behind a firewall in the organization’s data center. The Integration Gateway provides a secure, encrypted channel to transport this data to the Illuminate cloud environment, where Illuminate creates a secure Application Program Interface (API) access platform.

What is the current version of the Integration Gateway?

Illuminate Integration Gateway is at Version 4.0

*Note: Prior versions of the Illuminate Integration Gateway were branded "Illuminate Dataport"

What do we provide?

N2N provides a Docker image with the following software packages included:

  • Tomcat 9.0.20
  • JVM 11.0
  • Illuminate Integration Gateway Project 4.0 (the N2N application code)

General System Requirements

General system requirements for installing the Integration Gateway:

Software

  • Docker 17.x version or higher

Hardware (VM)

  • 4 to 8 CPU Cores (processors)

  • 8GB to 16GB RAM

  • Enough disk space to run the OS and other supporting software (recommended disk space 40+ GB)

Info:

Typically an 8 GB Ram server with 4 Core Processor can serve about 700-800 transactions per second. For more information about docker, prerequisites visit this Docker page


  • Typically an 8 GB Ram server with 4 Core Processor can serve about 700-800 transactions per second.

  • For more information about docker, prerequisites visit this Docker page

What is Docker and why does N2N use this?

What is Docker?

Docker is the world's leading software containerization platform. Docker containers wrap a piece of software in a complete file system that contains everything needed to run: code, run time, system tools, system libraries – anything that can be installed on a server. This guarantees that the software will always run the same, regardless of its environment. Containers isolate applications from one another and the underlying infrastructure while providing an added layer of protection for the application.

For more details about docker please visit Docker page

What is Docker Compose?

Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration.

For more details about docker compose please visit Docker Compose page

Why does N2N use Docker for the Illuminate Integration Gateway?

The Docker platform provides N2N with a simple, effective way to deliver the right package of software to the Illuminate Integration Gateway. Docker allows N2N to combine tested versions of software with validated configuration files into one deployment package. This process greatly reduces the number of manual steps, which improves reliability and robustness, as well as making deployment faster!

Docker Installation Instructions

Docker CE (Community Edition) is available on multiple platforms. Use the following matrix to choose a supported operating system that provides the best installation path for you. Click on the platform to navigate for detailed documentation on docker site. We provide detailed instructions for installing Docker and the Integration Gateway on CentOS along with other required configurations. Other Linux implementations will be similar.

Supported Platforms

Platform

Docket CE x86_64

CentOSYes (Recommended)

Ubuntu

Yes

Debian

Yes

Red Hat Enterprise Linux

Yes

Fedora

Yes

Microsoft Windows 10

Yes

macOS

Yes

Microsoft Azure

Yes

Amazon Web Services

Yes

Oracle Linux

Yes

SUSE Linux Enterprise Server

No

Microsoft Windows Server 2016

No


For more details please visit docker supported-platform documentation

Detailed instructions for installing Docker, Docker Compose and the Integration Gateway on CentOS

Prerequisites

  • An SSL certificate ready (self-signed is not accepted) which includes .crt file (the .crt file must have the public, root, and intermediate certs, with public cert on top of the file) and a private key. (See SSL Certs FAQs at the end of this document)

  • Ports 443, 80, 2376 (2376 port is optional) need to be open on the VM instance where Docker and Integration Gateway are installed

  • Get the authorization credentials from N2N to pull the Docker image

Docker Installation

There are two ways to install Docker Engine

  1. Install using the yum package manager (Recommended)
  2. Install using the instruction from https://docs.docker.com/install/linux/docker-ce/centos/

Install with yum

  • Log into your VM as a sudo or root user
  • Make sure your existing packages are up-to-date using yum
sudo yum update
  • Add the yum repository if it doesn’t exist on your VM

sudo tee /etc/yum.repos.d/docker.repo <<-'EOF' [dockerrepo] name=Docker Repository baseurl=https://yum.dockerproject.org/repo/main/centos/7/ enabled=1 gpgcheck=1gpgkey=https://yum.dockerproject.org/gpg EOF
  • Install the Docker package

sudo yum install docker-engine
  • Enable the service

sudo systemctl enable docker.service
  • Start the Docker daemon

sudo systemctl start docker
  • In order to make Docker service startup on server reboot run this command
sudo chkconfig docker on
  • Verify Docker has been installed correctly by running a test image in a container

docker ps
  • The screenshot below will show the installation is done properly.

Now that Docker installation is complete, the next step is to install Docker compose. 

Docker Compose Installation

  • Run this command to download the current stable release of Docker Compose

sudo curl -L "https://github.com/docker/compose/releases/download/1.24.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose


If you have problems installing with curl, see Alternate Installation Options


  • Apply executable permissions to the binary. 

sudo chmod +x /usr/local/bin/docker-compose


If the command docker-compose fails after installation, check your path. You can also create a symbolic link to /usr/binor any other directory in your path

For Example:

sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

For more details about Docker Compose, please visit here


  • Test the Installation

docker-compose --version

O/P: docker-compose version 1.24.0, build 1110ad01 it will show docker-compose version


Integration Gateway Installation

There are two ways to install Integration Gateway

  1. Install using the YAML script (Recommended)
  2. Install using manual steps

Integration Gateway Installation using YAML Script

  • Login as root on to the VM

  • Create a folder with the preferable name

For Example:
mkdir n2ndataport
  • Download the docker-compose.yml file which is provided by N2N

Once docker-compose up -d is executed following actions:

  • Default Docker network is created with the folder name where the docker-compose.yml file is located (in the above screenshot n2ndataport_default docker network is created based on folder n2ndataport)

  • Then in the next step, it started pulling the dataport image and Nginx image

  • In the final step, dataport and Nginx containers are created

  • Place this docker-compose.yml file in the folder 'n2ndataport' which was created in the above step

  • Run the docker-compose.yml file to deploy both Nginx and dataport Docker containers

docker-compose up -d


  • In the below screenshot, we can see that the dataport and Nginx containers are up and running.

Applying the SSL Certs
  • Now copy all your SSL certificate files (such as the .crt and .key files*) to the “certs” folder inside the Nginx container

docker cp <your file name>.crt nginx:/certs/<your file name>.crt 
docker cp <your file name>.key nginx:/certs/<your file name>.key     


File extensions .crt and .key are used as an example, use appropriate valid extensions as needed

  • Login to the Nginx container

docker exec -it nginx bash
  • Go to cd /etc/nginx/conf.d/

  • Open vim default.conf


  • Add "ssl" at line 2 as shown below

    • listen 443 ssl;

  • Change server_name to yourdnsname.com at line 3 and 22

  • Uncomment lines 4 and 5 (Do this step after copying certs to nginx container)

  • Now test whole Nginx config using this command

nginx -t
  • If the validation is successful, proceed to the next step, otherwise review the error and return to the Nginx default.conf file

if success enter "exit" commmand
  • Restart the Nginx container

service restart nginx
Test connection
  • Test the connection using curl. It should “return a response status as not a valid request”. Note, that you need to configure the command below to match organizational nomenclature

curl -X GET https://{server_name(or)host_name}/idp

Establishing a connection between Illuminate, Integration Gateway and the SIS Database

Step 1 - Connecting Illuminate and the Integration Gateway

In this step, we configure the Integration Gateway details within the Illuminate App. 

Step 2 - Connecting the Integration Gateway to the Database(s)

In this step, we will establish the connection between the Integration Gateway and the Database by creating a properties file and opening relevant ports.

Note: Within these instructions, EMPDB.properties is used as an example. EMPDB can be replaced with an appropriate database name. The database properties are stored in an encrypted format in EMPDB.properties.

Use the below method to configure the EMPDB.properties:

The properties are generated using a curl command. For security, please turn off Linux history before running the curl command, using the following set command:
set +o history
  • The Illuminate Integration Gateway includes a web service which will create these properties in the system.

  • There are four steps involved: 

    • Creating the request body with the database properties
    • Creating the cURL command
    • Add the database provider in the Illuminate app
    • Generating the token

1 - Creating the request body with the database properties

This service needs a JSON request body as shown below, edit this sample to include your specific database server information for Oracle or SQL Server:

  • A value should be populated for either SID or ServiceName but not both
  • Ensure there are no carriage return/line feed (CR/LF) characters in the parameters before running the full curl command


{
    "DBHost": "Server hostname or IP", // database host
    "port": "1234", // database port
    "SID": "SIDname", // sid or service name for oracle providers 
    "ServiceName": "servicename",
    "databaseName": "", // databaseName is for SQL server providers 
    "DBUsername": "name", // DB username or schema Name 
    "DBPassword": "***********" // DB password 
}


  • Example for Oracle Database

{
    "DBHost": "0.0.0.0",
    "port": "1234",
    "SID": "",
    "ServiceName": "servicename",
    "DBUsername": "username",
    "DBPassword": "***********"
}


  • Example for SQL Server Database

{
    "DBHost": "0.0.0.0",
    "port": "1234",
    "databaseName":  "",
    "DBUsername": "username",
    "DBPassword": "***********"
}
  • Example for Informix Database

{
  "DBHost": "0.0.0.0",
  "port": "1234",
  "databaseName": "",
  "DBUsername": "username",
  "DBPassword": "***********",
  "informixServer": ""
}

2 - Creating the cURL command

  • Run the following command to retrieve the Docker container IP (in this case we need the dataport container port)
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' dataport
  • Replace "dataport_container_ip:port" with the response body from the last step, in the following curl.  invoke the below service (Recommended)
curl -X POST 'http://{dataport_container_ip:port}/idp/addDBproperties' -H 'Authorization:Token from Illuminate connections page' -H 'Content-Type: application/json' -d '{"sourceName": "myDataSource","DBServer": "oracle","DBHost": "0.0.0.0","port": "1234","SID": "","ServiceName": "servicename","DBUsername": "username","DBPassword": "***********"}'
  • You can also use DNS (which is assigned to the dataport) to add DB properties file
curl -X POST 'https://{your_dns_name}/idp/addDBproperties' -H 'Authorization:Token from Illuminate connections page' -H 'Content-Type: application/json' -d '{"sourceName": "myDataSource","DBServer": "oracle","DBHost": "0.0.0.0","port": "1234","SID": "","ServiceName": "servicename","DBUsername": "username","DBPassword": "***********"}'


If the dataport is on a load balanced network and you use the DNS method to add the properties it will not update in all the containers. In this case, use the container IP to individually update the properties.

3 - Add the Database Provider in the Illuminate App

  • Ensure the Integration Gateway configuration step is completed
  • Login to the Illuminate App
  • Add the database provider details by following these instructions.  
    • An authorization token will be generated. This token is used to create or update the database properties i.e. the EMPDB.properties.

4 - Generating the token required for addDBproperties API

  • Once the provider is created in Illuminate and the token is generated, it is passed in the header of the above CURL command with Authorization as the key and token as the value. 
  • Note: You can connect to multiple databases by entering additional Providers in the Illuminate App connections module. Repeat the steps listed above to add providers, each additional provider requires its own generated token.
  • The connection information in the EMPDB.properties is encrypted.

Setup Link between Illuminate and Integration Gateway

Follow these instructions to complete the connectivity.

  • Login to the Illuminate App
  • Go to Connections → Setup Dataport
  • Acknowledge setup is complete
  • Enter Dataport details (Installation type = manual, Dataport Name and Dataport URL) 
  • Click Publish

Load Balancing the Integration Gateway (Recommended)

Load balancing refers to efficiently distributing incoming network traffic across a group of servers. A Load Balancer acts as the "traffic cop" sitting in front of your servers and routing requests across all servers capable of fulfilling those requests in a manner that maximizes speed and capacity utilization and ensures that no one server is overworked, which could degrade performance.If a single server goes down, the load balancer redirects traffic to the remaining online servers. When a new server is added to the server group, the load balancer automatically starts to send requests to it. 


If the Integration Gateway is on a load balanced network, please ensure appropriate steps are followed to update the database properties in all relevant servers/ container. 


For more details about Load Balancing

Sample Load Balancing Diagram