The data economy stands and falls on good data sharing systems. As a company, you probably want to focus on strategic decisions that will help shape the future of your business. What is our digital strategy? What revenue model is feasible and interesting to open up new markets? But have you thought about the technical part? How will your company share data, and what technology will make it easy and reliable to do so?
In this article, we will show you that technical choices about data sharing are an essential part of your digital strategy. Here we focus on the use of APIs, precisely because they offer so many benefits.
"APIs allow us to create products that give the user a seamless experience. The technology has to sit so correctly that it is not actually visible in the application."
API stands for "Application Programming Interface", an interactive link that allows two pieces of software or computer programs to communicate with each other. They are the foundation of the digitisation of our society today, and are often embedded in our daily lives and the applications we use, without us even realising it, such as itsme, payconiq or a meeting request via email that automatically appears in your calendar.
APIs are ubiquitous, extremely low-threshold and make integrating systems very easy. Developers can also use APIs to build applications much faster, which is very important if you want to keep up with the demands of your customers.
We are all familiar with interfaces that convert information from machines into something we humans can understand. Think of a text or image stored on a PC and displayed on a screen. APIs are interfaces that do exactly the same thing, but between two machines. An API allows two pieces of software to communicate as autonomously as possible. The information from one piece of software is converted by the API into information that another system can understand.
A well-known example is the use of payconiq. This app first checks autonomously with your bank whether the transaction can be carried out and then, if the answer is positive, carries it out. Or the app that allows you to turn on the heating at home before you arrive. The Google Maps API is used to integrate maps into all kinds of websites.
The software system that sends a command, e.g. a request for information such as "do you have parcel information for farmer X", is called the client. Through the API, the request is translated to a server, to a database, to software. Subsequently, another API can translate the client's request "give me the parcels of farmer X where potatoes are the main crop" into a command that the database can understand, so that it returns the correct parcels as an answer.
The above example also immediately illustrates the benefits of APIs. The customer can query the database without human intervention, so they always have the most up-to-date data. This is very different from, for example, sending an email to a person to get an export from a database. In addition, the information is delivered in a format that can be interpreted by software, so that it can be processed or stored autonomously if required.
Of course, since the advent of digitalisation, there is no longer a need to send an e-mail to get information from a database. Even before the advent of APIs, there were solutions for exchanging data.
A well-known and widely used technology is FTP (File Transfer Protocol). However, FTP is file-based, so you cannot give the customer a choice of which files they actually want. Only the files that have been prepared are available. This means that the FTP server has to be set up permanently, and the files have to be kept on the server. Because FTP is a file-based system, many ad hoc agreements must be made between the data provider and the user about the execution and encryption of the file content. To facilitate these agreements, standards/standard encoding EDI is often used for this purpose. A well-known example of this is the EDI standards available from Agroconnect, e.g. EDI dairy or EDIcrop. The standard describes the content and structure of the documents, eliminating the need for 1-to-1 agreements. Every chain partner speaks the same language and exchanges standard structured documents.
Other generations of technology have emerged over the years, often just to allow more flexible querying and data exchange. With the advent of the web, web APIs emerged, using web technology to create machine interactions. A REST API is a type of web API, where REST stands for REpresentation State Transfer, which refers to a particular type of software architecture built within a set of guidelines. You can read more about this in our Best Practice Guide "Exporting APIs".
Web APIs build on the web standards that are widely used to let two machines talk to each other. Many of the conventions needed to make this possible are commonplace, and an ever-growing group of specialists can build them. The structure of an API is also much simpler.
Another advantage of using APIs is that you can easily incorporate new versions of data. Since you don't know exactly when the data provider will make new data available, the data provider can let you know via events and an event bus. The data user can subscribe to published events, which will notify the user when the data provider sends out a new version. At that point, the data user can tap into the API to get the information.
Even if you create a new version of the API, you can very easily keep the previous version live, giving customers time to adapt, but new customers can already use the latest version. This does mean that as a company you need to have good version management.
When making an API available to (chain) partners, a number of things are crucial. Stephane Jans, In The Pocket, summarised them for us in a checklist that he explained in a webinar. Important aspects are: security, authentication, throttling, specifications, events, version control, consent management, ontology and standards.
The Flemish data sharing platform DjustConnect enables companies to share and use data securely and easily, while respecting the farmer, who is usually the data owner. DjustConnect is an API-based platform. Data providers can make data available via APIs in the ConnectShop, where interested parties can apply to use this data for a digital tool. The ConnectShop ensures that multiple interested parties find their way to your available API. But DjustConnect also facilitates the data exchange itself on a technical level, simply by helping companies to tick off the API checklist.
For example, DjustConnect takes care of your API management. Here DjustConnect takes care of the management of the API. This means that we provision your API, enforce the usage policy and control access. In addition, we collect metadata about the usage of the API so that you can see, for example, how often each data customer requests your API.
The checklist below tells you what to look out for and what else DjustConnect does to keep all users worry free.
|Checklist||Importance||How DjustConnect helps you|
|Security||Secure access to your data.||DjustConnect provides a secure connection and there is no direct contact between data provider and data consumer.|
|Authentication||Is the user really who they claim to be?||Data providers validate the incoming SSL client certificate from DjustConnect or OAuth2 is provided as authentication.|
|Throttling||Is there any protection on the system so that you can only get a maximum load of requests on your system?||In DjustConnect, the number of retrievals per time unit can be limited if desired.|
|Specifications||This is technical documentation for the developers.||OpenAPI specifications for the available APIs are made available on the DjustConnect developer portal.|
|Events (optional)||The data provider notifies consumers when new data is available, e.g. new lab analysis available.||DjustConnect offers the ability to use events to notify data consumers of updates to available data.|
|Version management||Each version of the API has its own version number.||DjustConnect allows new versions of an API to become available, but contractually enforces that the old version remains available to existing data customers for at least 3 months alongside the new version. In addition, DjustConnect has a strict policy regarding the transfer of consent between successive versions.|
|Consent management||Is the way the data owner's permissions on the use of the API are enforced.||Access to an API via DjustConnect can only take place after obtaining the appropriate permissions, in particular those of the API provider and - if the farmer owns the data - those of the farmer. Each farmer and data provider has a DjustConnect dashboard with an overview of all data requests, where they can grant or revoke these permissions. This mechanism is included in the legal contracts that each user accepts.|
|Ontology and standards||Data standards are documented agreements on representation, format, definition, structuring, transmission, manipulation, use and management of data. An ontology is a data model developed within a domain, describing all entities as well as representing the relationships between them.||DjustConnect does not impose an ontology. But ontology and standardisation make it much easier for customers to integrate data. As a result, they often increase the use and thus the value of the API. That is why we like to explore existing standards and opportunities together.|
By offering these services, we want to take data sharing in the Flemish agri-food sector one step further. DjustConnect facilitates this, but it is up to the companies to develop the APIs.
Want to read more? Check out the Best Practice Guide "My API on DjustConnect".