Skip to content

Conversation

@jotelha
Copy link
Member

@jotelha jotelha commented Sep 21, 2022

This is an old pull request I had still open on Luis' fork (ljyanesm#1), with a few points not discussed yet. Below I copy over what I still find relevant. A few of the issues described arose when playing with auto-generating Python API from the openapi specs (https://github.com/jotelha/dtool-lookup-server/tree/0de6f54d0912c79740c9089228f2c5c764babf26/devel).

ERROR    dtool_lookup_server:app.py:1455 Exception on /dataset/register [POST]
Traceback (most recent call last):
  File "/mnt/dat2/git/dtool/dtool-lookup-server/venv/lib/python3.8/site-packages/flask/app.py", line 2077, in wsgi_app
    response = self.full_dispatch_request()
  File "/mnt/dat2/git/dtool/dtool-lookup-server/venv/lib/python3.8/site-packages/flask/app.py", line 1525, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/mnt/dat2/git/dtool/dtool-lookup-server/venv/lib/python3.8/site-packages/flask_cors/extension.py", line 165, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/mnt/dat2/git/dtool/dtool-lookup-server/venv/lib/python3.8/site-packages/flask/app.py", line 1523, in full_dispatch_request
    rv = self.dispatch_request()
  File "/mnt/dat2/git/dtool/dtool-lookup-server/venv/lib/python3.8/site-packages/flask/app.py", line 1509, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/mnt/dat2/git/dtool/dtool-lookup-server/venv/lib/python3.8/site-packages/webargs/core.py", line 452, in wrapper
    return func(*args, **kwargs)
  File "/mnt/dat2/git/dtool/dtool-lookup-server/venv/lib/python3.8/site-packages/flask_smorest/arguments.py", line 82, in wrapper
    return func(*f_args, **f_kwargs)
  File "/mnt/dat2/git/dtool/dtool-lookup-server/venv/lib/python3.8/site-packages/flask_smorest/response.py", line 90, in wrapper
    func(*args, **kwargs)
  File "/mnt/dat2/git/dtool/dtool-lookup-server/venv/lib/python3.8/site-packages/flask_jwt_extended/view_decorators.py", line 127, in decorator
    return current_app.ensure_sync(fn)(*args, **kwargs)
  File "/mnt/dat2/git/dtool/dtool-lookup-server/dtool_lookup_server/dataset_routes.py", line 148, in register
    dataset_uri = register_dataset(dataset_info)
  File "/mnt/dat2/git/dtool/dtool-lookup-server/dtool_lookup_server/utils.py", line 686, in register_dataset
    register_dataset_descriptive_metadata(dataset_info.copy())
  File "/mnt/dat2/git/dtool/dtool-lookup-server/dtool_lookup_server/utils.py", line 632, in register_dataset_descriptive_metadata
    _register_dataset_descriptive_metadata(collection, dataset_info)
  File "/mnt/dat2/git/dtool/dtool-lookup-server/dtool_lookup_server/utils.py", line 658, in _register_dataset_descriptive_metadata
    exists = collection.find_one(query)
  File "/mnt/dat2/git/dtool/dtool-lookup-server/venv/lib/python3.8/site-packages/pymongo/collection.py", line 1419, in find_one
    for result in cursor.limit(-1):
  File "/mnt/dat2/git/dtool/dtool-lookup-server/venv/lib/python3.8/site-packages/pymongo/cursor.py", line 1248, in next
    if len(self.__data) or self._refresh():
  File "/mnt/dat2/git/dtool/dtool-lookup-server/venv/lib/python3.8/site-packages/pymongo/cursor.py", line 1165, in _refresh
    self.__send_message(q)
  File "/mnt/dat2/git/dtool/dtool-lookup-server/venv/lib/python3.8/site-packages/pymongo/cursor.py", line 1052, in __send_message
    response = client._run_operation(
  File "/mnt/dat2/git/dtool/dtool-lookup-server/venv/lib/python3.8/site-packages/pymongo/mongo_client.py", line 1267, in _run_operation
    return self._retryable_read(
  File "/mnt/dat2/git/dtool/dtool-lookup-server/venv/lib/python3.8/site-packages/pymongo/mongo_client.py", line 1371, in _retryable_read
    return func(session, server, sock_info, read_pref)
  File "/mnt/dat2/git/dtool/dtool-lookup-server/venv/lib/python3.8/site-packages/pymongo/mongo_client.py", line 1263, in _cmd
    return server.run_operation(
  File "/mnt/dat2/git/dtool/dtool-lookup-server/venv/lib/python3.8/site-packages/pymongo/server.py", line 100, in run_operation
    message = operation.get_message(read_preference, sock_info, use_cmd)
  File "/mnt/dat2/git/dtool/dtool-lookup-server/venv/lib/python3.8/site-packages/pymongo/message.py", line 375, in get_message
    request_id, msg, size, _ = _op_msg(
  File "/mnt/dat2/git/dtool/dtool-lookup-server/venv/lib/python3.8/site-packages/pymongo/message.py", line 673, in _op_msg
    return _op_msg_uncompressed(flags, command, identifier, docs, opts)
  File "/mnt/dat2/git/dtool/dtool-lookup-server/venv/lib/python3.8/site-packages/bson/binary.py", line 267, in from_uuid
    raise ValueError(
ValueError: cannot encode native uuid.UUID with UuidRepresentation.UNSPECIFIED. UUIDs can be manually converted to bson.Binary instances using bson.Binary.from_uuid() or a different UuidRepresentation can be configured. See the documentation for UuidRepresentation for more information.

treat UUIDs as string for now, keep https://pymongo.readthedocs.io/en/stable/examples/uuid.html#handling-uuid-data in mind.

jotelha and others added 25 commits April 12, 2022 11:30
…a in sql_models.py and schmemas.py caused troubles for autogenerated Python client API.

  For now, resorted to suboptimal suffix "SQLAlchemy" for the schemata defined within sql_models.py.
  Do we need actually need the different schema in sql_models.BaseURISQLAlchemySchema(ma.SQLAlchemyAutoSchema) and schemas.class BaseURISchema(Schema) ?
* sql_models.Dataset(db.Model) defines columns frozen_at and created_at as db.DateTime(), its method as_dict() converts those with
  dtoolcore.utils.timestamp(self.frozen_at). This, however, results in the autogenerated client API to expect a serialized datetime object and
  throw an exception when receiving float. For now, inverted the previos behavior of the server to always return timestamps as floats to
  returning datetime objects instead. What should be the way here in the future?
* Standardized spelling of Uri to URI in models and schema for all occurences.
@jotelha jotelha changed the title Bits and pieces on openapi implementation WIP: Bits and pieces on openapi implementation Sep 21, 2022
@jotelha
Copy link
Member Author

jotelha commented Oct 19, 2022

An instance of this branch runs at https://demo.dtool.dev/, with openapi docs at https://demo.dtool.dev/lookup/doc/swagger, would like to discuss.

@jotelha jotelha changed the title WIP: Bits and pieces on openapi implementation Bits and pieces on openapi implementation Oct 19, 2022
This was referenced Nov 13, 2022
@jotelha jotelha mentioned this pull request Nov 20, 2022
@jotelha
Copy link
Member Author

jotelha commented Nov 20, 2022

@jotelha jotelha closed this Nov 20, 2022
@jotelha jotelha deleted the openapi branch November 23, 2022 09:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant