With data coming in from many sources and distributed across different stores, it's getting tougher to find the right data and to know how to use it.
On a low level there is for example the "comment on" feature of SQL which won't help with other stores like AWS S3 and thus cannot be a generic solution. Then there are data catalog tools like Azure Data Catalog or Collibra Catalog which seem to be geared towards business users and not as much to developers.
I think a good solution needs to support discovery of data (e.g. through searchability and tagging) and allow to explain how to interpret the data (e.g. through detailed comments).
What are your best practices or recommendations on tools to use?
On a low level there is for example the "comment on" feature of SQL which won't help with other stores like AWS S3 and thus cannot be a generic solution. Then there are data catalog tools like Azure Data Catalog or Collibra Catalog which seem to be geared towards business users and not as much to developers.
I think a good solution needs to support discovery of data (e.g. through searchability and tagging) and allow to explain how to interpret the data (e.g. through detailed comments).
What are your best practices or recommendations on tools to use?