Hi folks,
I’m looking into doing some post-processing of the manifest.json, however once I’ve edited the manifest file and run dbt docs serve the manifest.json is reverted to the pre-edit state. Does anyone know what could be causing that and how to prevent it?
Concretely I’m adding hyperlinks to model description based on metadata.
Are you sure it is dbt docs serve that is overwriting manifest.json? Or could it be the dbt docs generate?
dbt docs generate command compiles the project, causing the manifest.json to change, see Step (2) here About dbt docs commands | dbt Developer Hub
If you want to skip re-compilation you can use the flag --no-compile
dbt docs generate --no-compile
If it is indeed the dbt docs serve then I have no answer for that 
I would expect dbt docs generate to change the manifest.json but that isn’t the case.
Steps to replicate:
Expected result: No modification to the manifest.json and the manual edits piped to the dbt docs UI
Observed result: manifest.json reverted to before step 2 and the edits not appearing in the dbt docs UI. Additionally index.html also seems to have been modified but the catalog.json are unchanged.
I’m using dbt-core and dbt-bigquery both v1.5.0 if that’s matters.
That is strange, just tested with dbt-bigquery v1.5.1 and the manifest.json didn’t change when running dbt docs serve
which adapter are you using?
That is strange indeed. I just tested it again and still observe the same behaviour, even without editing the manifest.json.
I think I’m using bigquery.
dbt docs serve won’t modify the json files. Something else must be causing the files to be reverted (or not to be saved in the first place).
How are you modifying the file? Is it with a Python script or are you editing it manually? Have you validated that the script has read-write permissions to the file? Once you edit and save the file, can you open it up in a text editor and ensure that the changes have applied correctly?
It might also be worth checking that you’re not serving the docs at the same time as you edit them; it might not let you save changes if another process already has the file open.
To try to isolate the problem further I just did.
This changed the ‘last modified’ timestamp on index.html, partial_parse.msgpack, and manifest.json. I took no other actions.
To address your questions I am planning to use a Python script eventually but to reduce complexity I’m currently manually editing it in an editor. I’ve confirmed that the save is successful. To validate that further I made a copy of the manifest.json, confirmed that the edit also existed in the duplicate. After running dbt docs serve --port 8000 I notice the manifest.json being reverted back to the pre-edit version but the duplicate remains untouched.
That’s really weird! Could I get you to open an issue here?
I’d be surprised if there’s anything in a patch release, but the fact that you were using 1.5.0 and Bruno was using 1.5.1 is the only difference I can see. It could be worth making sure you’re up to date on patches to dbt Core (and dbt-bigquery for the sake of completeness, but it shouldn’t have anything to do with writing json files) before filing the issue.
dbt 1.6 is out today and I think this was fixed by Do not rewrite `manifest.json` during `docs serve` command by jtcohen6 · Pull Request #7554 · dbt-labs/dbt-core · GitHub - would you mind giving it a go?
Edit: it should actually be available from 1.5.1 and onwards!
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.