Do The Git Repository Data Structures Use A Canonical Encoding?

I'm using dulwich (a Python library) to access a git repository. When I use get_object to retrieve a commit, it has a number of attributes. One of those is author. When I retrieve

Solution 1:

Metadata is supposed to be encoded with the value set by the i18n.commitEncoding config value; whenever a commit is created the current value is copied into the 'encoding' header on the object, if set; the default value is UTF-8.

That encoding value is available on Dulwitch objects as the '.encoding' attribute; if it is None then i18n.commitEncoding was not explicitly set and you can use UTF-8 as the default.

However! The actual data stored simply follows whatever bytes where handed to git and no re-coding takes place. The configuration value is purely informational. So you need to take into account that an incorrect codec was used, if you are going to use object.encoding or 'utf8' as the codec, use a sensible error handler or fallback strategy.