I can totally relate to what you describe here ... I have never found the lineage representations in the metadata tools to be very helpful. It feel like gimmickry to convince management but not the solution you need in your engineer's toolbelt to effectively navigate the complexity of your data reality. A search based tool with data lineage operators/verbs is indeed the solution ("search over navigation", right?).
But would it be so simple to dump the dataset in a graphdb like Neo4j and be over with it?
I can totally relate to what you describe here ... I have never found the lineage representations in the metadata tools to be very helpful. It feel like gimmickry to convince management but not the solution you need in your engineer's toolbelt to effectively navigate the complexity of your data reality. A search based tool with data lineage operators/verbs is indeed the solution ("search over navigation", right?).
But would it be so simple to dump the dataset in a graphdb like Neo4j and be over with it?