In my second professional article, I describe how metadata-driven design can be employed to build an ingestion system for data retrieved through an API. I must give many thanks to the staff at InfoQ for taking the time to help with this submission. If you have any interest or stake in enterprise-level data ingestion, there’s a good chance that you’ll find something of value in it. Go check it out!

——–

Metadata-Driven Design: Designing a Flexible Engine for API Data Retrieval
Bulk data is commonly accessed via files & FTP. As the world moves toward APIs to facilitate collaboration, what are the requirements for data APIs? In this article, Aaron Kendall of Barnes & Noble describes a meta-data driven architecture for bulk data ingestion. Two APIs operate in parallel to provide data changes as well as the data records themselves. An example demonstrates how API responses are parameterized using meta-data.

On a fortunate note, InfoQ editor Saul Caganoff was kind enough to point out definitive proof that metadata-driven design has had multiple first adopters. Over at MSDN, it seems that an old article by Kevin Perera comes to similar conclusions about the benefits of MDD. It’s especially funny to think that nearly a decade ago, two people were having the same thoughts on different sides of the planet…and even gave it the same name! In any case, if you have any interest in this subject at all, you should definitely head over there and check it out.