What is the Illuminate Integration Gateway?

The Illuminate Integration Gateway, also called the Dataport, is a tool used to provide the Illuminate platform with access to data that is located in physically and logically secure configurations, such as behind a firewall in the organization’s data center. The Integration Gateway provides a secure, encrypted channel to transport this data to the Illuminate cloud environment, where Illuminate creates a secure Application Program Interface (API) access platform.

What is the current version of the Integration Gateway?

Illuminate Integration Gateway is at Version 4.0

*Note: Prior versions of the Illuminate Integration Gateway were branded "Illuminate Dataport"

What do we provide?

N2N provides a Docker image with the following software packages included:

  • Tomcat 9.0.20
  • JVM 11.0
  • Illuminate Integration Gateway Project 4.0 (the N2N application code)

General System Requirements

General system requirements for installing the Integration Gateway:

Software

  • Docker 17.x version or higher

Hardware (VM)

  • 4 to 8 CPU Cores (processors)

  • 8GB to 16GB RAM

  • Enough disk space to run the OS and other supporting software (recommended disk space 40+ GB)


  • Typically an 8 GB Ram server with 4 Core Processor can serve about 700-800 transactions per second.

  • For more information about docker, prerequisites visit this Docker page

What is Docker and why does N2N use this?

What is Docker?

Docker is the world's leading software containerization platform. Docker containers wrap a piece of software in a complete file system that contains everything needed to run: code, run time, system tools, system libraries – anything that can be installed on a server. This guarantees that the software will always run the same, regardless of its environment. Containers isolate applications from one another and the underlying infrastructure while providing an added layer of protection for the application.

For more details about docker please visit Docker page

Why does N2N use Docker for the Illuminate Integration Gateway?

The Docker platform provides N2N with a simple, effective way to deliver the right package of software to the Illuminate Integration Gateway. Docker allows N2N to combine tested versions of software with validated configuration files into one deployment package. This process greatly reduces the number of manual steps, which improves reliability and robustness, as well as making deployment faster!

Docker Installation Instructions

Docker CE (Community Edition) is available on multiple platforms. Use the following matrix to choose a supported operating system that provides the best installation path for you. Click on the platform to navigate for detailed documentation on docker site. We provide detailed instructions for installing Docker and the Integration Gateway on CentOS along with other required configurations. Other Linux implementations will be similar.

Supported Platforms

Platform

Docket CE x86_64

CentOSYes (Recommended)

Ubuntu

Yes

Debian

Yes

Red Hat Enterprise Linux

Yes

Fedora

Yes

Microsoft Windows 10

Yes

macOS

Yes

Microsoft Azure

Yes

Amazon Web Services

Yes

Oracle Linux

Yes

SUSE Linux Enterprise Server

No

Microsoft Windows Server 2016

No


For more details please visit docker supported-platform documentation

Detailed instructions for installing Docker and the Integration Gateway on CentOS

Prerequisites

  • An SSL certificate ready (self-signed is not accepted) which includes .crt file (the .crt file must have the public, root, and intermediate certs, with public cert on top of the file) and a private key. (See SSL Certs FAQs at the end of this document)

  • Ports 443, 80, 2376 (2376 port is optional) need to be open on the VM instance where Docker and Integration Gateway are installed

  • Get the authorization credentials from N2N to pull the Docker image

Docker Installation

There are two ways to install Docker Engine

  1. Install using the yum package manager (Recommended)
  2. Install using the instruction from https://docs.docker.com/install/linux/docker-ce/centos/

Install with yum

  • Log into your VM as a sudo or root user
  • Make sure your existing packages are up-to-date using yum
sudo yum update
  • Add the yum repository if it doesn’t exist on your VM

sudo tee /etc/yum.repos.d/docker.repo <<-'EOF' [dockerrepo] name=Docker Repository baseurl=https://yum.dockerproject.org/repo/main/centos/7/ enabled=1 gpgcheck=1gpgkey=https://yum.dockerproject.org/gpg EOF
  • Install the Docker package

sudo yum install docker-engine
  • Enable the service

sudo systemctl enable docker.service
  • Start the Docker daemon

sudo systemctl start docker
  • In order to make Docker service startup on server reboot run this command
sudo chkconfig docker on
  • Verify Docker has been installed correctly by running a test image in a container

docker ps
  • The screenshot below will show the installation is done properly.

Integration Gateway Installation

Integration Gateway installation using manual steps (Without YAML Script)

  • Login to the VM instance as root

  • Create a new Docker network using the following commands

docker network create --driver bridge dataportnetwork
  • In order to pull N2N’s Integration Gateway image from the N2N private repository (quay.io) please enter Docker login credentials separately provided by N2N 

For Example:
docker login -u="n2ndevops+n2n" -p="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" quay.io
  • Pull and Run N2N's Integration Gateway image
/* For the latest versions of Oracle (12.2 and above) pull the illuminate Dataport V4.0 */

docker pull quay.io/n2ndevops/illuminatedataport:4.0

/* To Run this dataport image */

docker run --network=dataportnetwork --net-alias dataport --name dataport -it -d quay.io/n2ndevops/illuminatedataport:4.0


/* For Oracle versions below (12.1 and below), please pull the Illuminate Dataport V3.9.1 (note that version 4.0.1 was a special maintenance release with a down level version of the JDBC driver) */

docker pull quay.io/n2ndevops/illuminatedataport:4.0.1

/* To Run this dataport image */

docker run --network=dataportnetwork --net-alias dataport --name dataport -it -d quay.io/n2ndevops/illuminatedataport:4.0.1
  • Pull and run the nginx Docker image (This is a public image verified by Docker hub that has been cached by N2N for consistent versioning and performance)

/* Image */

docker pull quay.io/n2ndevops/nginx:1.16

/* Run nginx image */

docker run --name nginx --network=dataportnetwork -it -p 80:80 -p 443:443 -d quay.io/n2ndevops/nginx:1.16
  • Check to ensure both containers are running properly, indicating the “running” state

docker ps


Applying the SSL Certs
  • Now copy all your SSL certificate files (such as the .crt and .key files*) to the “certs” folder inside the nginx container

docker cp <your file name>.crt nginx:/certs/<your file name>.crt 
docker cp <your file name>.key nginx:/certs/<your file name>.key     
  • Login to the Nginx container

docker exec -it nginx bash
  • Go to cd /etc/nginx/conf.d/

  • Open vim default.conf


  • Add "ssl" at line 2 as shown below

    • listen 443 ssl;

  • Change server_name to yourdnsname.com at line 3 and 22

  • uncomment 4 and 5 lines (Do this step after copying certs to nginx container)

  • Now test whole Nginx config using this command

nginx -t
  • If the validation is successful, proceed to the next step, else review the error and return to the Nginx default.conf file

if success enter "exit" command
  • Restart  Nginx container

service restart nginx
Test connection
  • Test the connection using curl. It should “return a response status as not a valid request”. Note that you need to configure the command below to match organizational nomenclature

curl -X GET https://{server_name(or)host_name}/idp

Establishing a connection between Illuminate, Integration Gateway and the SIS Database

Step 1 - Connecting Illuminate and the Integration Gateway

In this step, we configure the Integration Gateway details within the Illuminate App. 

Step 2 - Connecting the Integration Gateway to the Database(s)

In this step, we will establish the connection between the Integration Gateway and the Database by creating a properties file and opening relevant ports.

Note: Within these instructions, EMPDB.properties is used as an example. EMPDB can be replaced with an appropriate database name. The database properties are stored in an encrypted format in EMPDB.properties.

Use the below method to configure the EMPDB.properties:

The properties are generated using a curl command. For security, please turn off Linux history before running the curl command, using the following set command:
set +o history
  • The Illuminate Integration Gateway includes a web service which will create these properties in the system.

  • There are four steps involved: 

    • Creating the request body with the database properties
    • Creating the cURL command
    • Add the database provider in the Illuminate app
    • Generating the token

1 - Creating the request body with the database properties

This service needs a JSON request body as shown below, edit this sample to include your specific database server information for Oracle or SQL Server:

  • A value should be populated for either SID or ServiceName but not both
  • Ensure there are no carriage return/line feed (CR/LF) characters in the parameters before running the full curl command


{
    "DBHost": "Server hostname or IP", // database host
    "port": "1234", // database port
    "SID": "SIDname", // sid or service name for oracle providers 
    "ServiceName": "servicename",
    "databaseName": "", // databaseName is for SQL server providers 
    "DBUsername": "name", // DB username or schema Name 
    "DBPassword": "***********" // DB password 
}


  • Example for Oracle Database

{
    "DBHost": "0.0.0.0",
    "port": "1234",
    "SID": "",
    "ServiceName": "servicename",
    "DBUsername": "username",
    "DBPassword": "***********"
}


  • Example for SQL Server Database

{
    "DBHost": "0.0.0.0",
    "port": "1234",
    "databaseName":  "",
    "DBUsername": "username",
    "DBPassword": "***********"
}
  • Example for Informix Database

{
  "DBHost": "0.0.0.0",
  "port": "1234",
  "databaseName": "",
  "DBUsername": "username",
  "DBPassword": "***********",
  "informixServer": ""
}

2 - Creating the cURL command

  • Run the following command to retrieve the Docker container IP (in this case we need the dataport container port)
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' dataport
  • Replace "dataport_container_ip:port" with the response body from the last step, in the following curl.  invoke the below service (Recommended)
curl -X POST 'http://{dataport_container_ip:port}/idp/addDBproperties' -H 'Authorization:Token from Illuminate connections page' -H 'Content-Type: application/json' -d '{"sourceName": "myDataSource","DBServer": "oracle","DBHost": "0.0.0.0","port": "1234","SID": "","ServiceName": "servicename","DBUsername": "username","DBPassword": "***********"}'
  • You can also use DNS (which is assigned to the dataport) to add DB properties file
curl -X POST 'https://{your_dns_name}/idp/addDBproperties' -H 'Authorization:Token from Illuminate connections page' -H 'Content-Type: application/json' -d '{"sourceName": "myDataSource","DBServer": "oracle","DBHost": "0.0.0.0","port": "1234","SID": "","ServiceName": "servicename","DBUsername": "username","DBPassword": "***********"}'


If the dataport is on a load balanced network and you use the DNS method to add the properties it will not update in all the containers. In this case, use the container IP to individually update the properties.

3 - Add the Database Provider in the Illuminate App

  • Ensure the Integration Gateway configuration step is completed
  • Login to the Illuminate App
  • Add the database provider details by following these instructions.  
    • An authorization token will be generated. This token is used to create or update the database properties i.e. the EMPDB.properties.

4 - Generating the token required for addDBproperties API

  • Once the provider is created in Illuminate and the token is generated, it is passed in the header of the above CURL command with Authorization as the key and token as the value. 
  • Note: You can connect to multiple databases by entering additional Providers in the Illuminate App connections module. Repeat the steps listed above to add providers, each additional provider requires its own generated token.
  • The connection information in the EMPDB.properties is encrypted.

Setup Link between Illuminate and Integration Gateway

Follow these instructions to complete the connectivity.

  • Login to the Illuminate App
  • Go to Connections → Setup Dataport
  • Acknowledge setup is complete
  • Enter Dataport details (Installation type = manual, Dataport Name and Dataport URL) 
  • Click Publish

Load Balancing the Integration Gateway (Recommended)

Load balancing refers to efficiently distributing incoming network traffic across a group of servers. A Load Balancer acts as the "traffic cop" sitting in front of your servers and routing requests across all servers capable of fulfilling those requests in a manner that maximizes speed and capacity utilization and ensures that no one server is overworked, which could degrade performance.If a single server goes down, the load balancer redirects traffic to the remaining online servers. When a new server is added to the server group, the load balancer automatically starts to send requests to it. 


If the Integration Gateway is on a load balanced network, please ensure appropriate steps are followed to update the database properties in all relevant servers/ container. 


For more details about Load Balancing

Sample Load Balancing Diagram