Fix a few casts where ints were cast to uints unnecessarily

Fixes #199. Previously, enums were treated as uint32's, but they need to
be treated as int32's instead.

t: Add a few test cases with negative enum values