Operator APIs | Data Ingestion
Data Ingestion is a product that allows NetRefer clients to use REST principles to ingest data into the PMP.
It is possible to ingest data in real-time sending a single event or in batch.
To access the APIs, clients should have an appropriate account registered with NetRefer.
This account provides access to specific NetRefer products, which in our case is API.
Also, clients must provide a Subscription Key allowing access to Data Ingestion APIs.
The authentication flow is depicted conceptually in the diagram below.
Using Active Directory to request a JWT
OAuth 2.0 is used to get access to the Data Ingestion API.
To authenticate within NetRefer and to call the Data Ingestion API, developers must request JWT, providing the following parameters:
Grant type: password
Client ID: will be provided to the client while onboarding to Data Ingestion. Example of id is: 5a324196-7727-45fd-8ac3-1436cd541d76
Access Token URL: https://login.microsoftonline.com/0e0b50ac-5253-4347-b528-251b2f17cfc6/oauth2/v2.0/token
Client Secret: will be provided to the client while onboarding to Data Ingestion
Username: will be provided to the client while onboarding to Data Ingestion
Password: will be provided to the client while onboarding to Data Ingestion
Once all the requirements are met, execute an HTTP POST call to "Access Token URL" to get the JWT.
In the API request, you have to add an "Authorization" header with the value:
Bearer JWT-value-from-NetRefer-Data Ingestion
An example of a request using different languages can be found on the APIs page of this portal.
Authorization to use Data Ingestion product
To authenticate access to the Data Ingestion product, HTTP requests must include an additional header:
Ocp-Apim-Subscription-Key: will be provided to the client during onboarding to Data Ingestion. Example: baea25a9f0fe4687845575c07e539a9e9
A full example of the Data Ingestion API call is illustrated in the diagram below.
All Data Ingestion APIs are exposed as REST endpoints.
Request and response models
Request data models are presented and described via the OpenAPI specification.
These specifications can be downloaded from the portal and the developers can build their own code proxies.
In the Developer Portal, you can also find an example of request\response objects and their schema:
Response Status Codes
Learn more about common Data Ingestion Developer error codes and how to resolve them.
Error codes describe events’ validation statuses of the DATA INGESTION API.
If you don't find an error code listed here, please refer to the API-specific error codes in their respective documentation pages.
All events are sent in batches but processed individually. If all events are valid, then 200 will be returned. If some of the events are not valid, then 207 will be returned. If all the events are not valid, then 400 will be returned.
Additional validation for all event types:
multiple events with the same EventID must not be processed
JSON with unproper multiple structures must not be processed
Data Ingestion is designed on the most cutting-edge technology stack and platform.
The API is constantly available, allowing for real-time data input.
If the API is unavailable for any reason, or if the status codes received are one of the following, Data Ingestion customers should follow these guidelines:
Contact your consultant and/or NetRefer Support if you have 401, 403, 404, or 500 (if it appears for a long period of time).
We recommend using the retry strategy to trigger your endpoints until they succeed in providing real-time data intake (and to be proactive with 500 error code processing). There may be a one-, three-, five-, or ten-minute time gap between calls, or any other interval the customer chooses.