[sohailshk]

Rationale for this change

A server that serializes unset fields as null makes the table and view create/load/register responses fail with a pydantic validation error, because TableResponse.config, TableResponse.storage_credentials and ViewResponse.config only default when the key is absent. ConfigResponse has the same problem for defaults and overrides: RestCatalog raises AttributeError: 'NoneType' object has no attribute 'update' while fetching the config on construction.

Java reads a null value as absent: LoadTableResponseParser uses hasNonNull for config and storage-credentials, ConfigResponseParser does the same for defaults/overrides, and LoadViewResponseParser was aligned in apache/iceberg#17250. This adds the same before-validators that NamespaceResponse.properties already has, so null reads as {} / [].

Not covered here: the scan planning models in scan_planning.py (null delete-files / file-scan-tasks / plan-tasks, which #17250 also fixed on the Java side), left as a follow-up.

Are these changes tested?

Yes: test_load_responses_with_null_config_and_storage_credentials, test_config_response_with_null_defaults_and_overrides and test_catalog_with_null_config_defaults_and_overrides in tests/catalog/test_rest.py (all fail on main, pass here). tests/catalog/test_rest.py and tests/catalog/test_rest_auth.py pass.

Are there any user-facing changes?

Tables and views can be loaded, and the catalog constructed, against REST servers that return explicit nulls for these fields.

[rambleraptor]

This is very much a personal thought, so I'm happy to be overridden.

[rambleraptor]

This feels really messy. I'm personally not a huge fan of field_validators being appended onto models, since it can be hard to understand the control flow.

What if we created a new type like NullableList that has this property built in. Then, we can just change the Field type. As a bonus, we can then reuse it throughout.

It would look something like:

def _empty_if_none(factory: Callable[[], Any]) -> BeforeValidator:
    return BeforeValidator(lambda v: factory() if v is None else v)

# A JSON null is read as an empty map / list, like Java's `hasNonNull`
NullableList = Annotated[list[T], _empty_if_none(list)]

[sohailshk]

Agreed, that is easier to follow. Done in c76e032: added NullableList and NullableProperties (an Annotated type with a BeforeValidator that turns null into an empty list / map) and switched the fields to them, so the four field_validators are gone. ConfigResponse.defaults and overrides are now typed Properties instead of Properties | None, since a null already came back as {}. The REST catalog tests still pass.