Merge FairSync and InCommon or make it interoperable

@yala I do not really understand what you mean, IN COMMON is suppose to be the base to foster this internationalisation by means of sharing the data.

I used the term internationalisation here in the technical meaning of translating the user interface strings. All the elements are written in french, and I didn’t find an option to switch the locale. Adding a layer of internationalisation to the interface would allow to translate it in many different languages.

Update about fairsync:

  • For Synchronization we want to use calDAV as a widely spread standard:
    • calDAV for Events (which is interoperable with Nextcloud)
    • cardDAV for Locations (which builds on vCard)

ActivityPub will become interesting for us a bit later, when it comes to spread news and ratings.

Main part of fairsync will be a microservice to detect and handly duplicates.
This microservice can also provide an activity pub endpoint.

I have created an overview sheet, comparing datafields of openfairDB and vCard, which maches very well: https://cloud.m4h.network/index.php/s/BXbR2k6n4MQXsX3
As soon as your fields at inCommon are documented, I will add them too.

@how @natacha: Do you mind just orientating the fields of in Common along the speicifaction of vCards for locations and calDAV for Events?

We would very much love to be you delegate for the first version of data syncronization. Could you please send me a file where I can see, how gogocarto-data is structured?

1 Like

*DAV compatibility would be interesting. I need to do a bit of research on this, but if people are interested, it could become part of a milestone. Yet ActivityPub has probably more priority for IN COMMON. I guess @ekes from Radar would have an opinion regarding iCal data format, as suggested in a previous discussion we had about Mobilizon and alternatives.

I’ll try to do the same and see where IN COMMON stands.

1 Like

Amazing this makes sense, we certainly could integrate the results. IN COMMON will encourage rich of information, to feed a relational model.

I think you should ask this directly to @Sebastian but I generally think it is documented here: https://pixelhumain.github.io/GoGoCartoJs/docs/dataset-element-configuration.html

I agree with @how this is important conversation that we want to continue having especially as Mobilizon is developping, and yes the conversation about Ical

1 Like

Thank you for the nice overview of possibilites.

I think it’s good here to separate between the wire protocol (DAV, based on XML), and the vocabulary used to depict the objects in the serialization. Where vCard is an accepted standard, synchronisation with an XML-based protocol will always come at the downside, that it is not accessible by browser clients.

This would be a strange decision, given that we are moving more and more logic into the browser clients, and render our servers dumb storage engines, connected with a JSON-via-HTTP protocol.

I am not sure if I would be willing going down the road of reviving XML-based standards for the core of a synchronisation logic, if we have JSON(-LD) available and at hands, especially for new developments.

If people wanted to have access to their events via a DAV endpoint, we can always publish an iCal feed, which people may add to their Nextcloud as well, and have it accessible from there.

In general this conversation seems more about which vocabularies for events (iCal? schema.org?) and locations (vCard? schema.org?) exist, and how we can map between them.

Secondly the means of synchronisation and federation, the protocol, becomes a second degree question for interoperability (neccessary condition), if the first is answered (sufficient condition).

2 Likes

I believe this is a key insight. Separation of data semantics from data serialization.

This has allowed a project I am working on (openEngiadina) to directly use events from Radar with zero-coordination. The trick is that Radar uses a well known collection of properties and classes (schema_org) and has the content embedded on the Website (using RDFa).

Semantics are collections of properties and classes. These properties and classes are defined independent of the context they are used in, independent of data representation format and independent of transport protocols.

There are many existing collections of properties and classes to describe things - these collections are called vocabularies or ontologies. Examples include schema.org or even ActivityStreams, which defines the properties and classes used in ActivityPub.

Data serialization formats include JSON-LD (as used in ActivityPub), Turtle, XML, embeddings in HTML. Choose whatever fits the client best, the content itself is the same.

It turns out that ActivityPub is basically a protocol for synchronizing such data (Linked Data) from client to server and server to server. Maybe that’s a way to make FairSync and InCommon (and openEngiadina and the Fediverse and many existing data repositories) interoperable?

2 Likes

ActivityPub ist still our vision.
Today we had a first call after along break to pic up this topic.
Our first mvp is to connect wechange.de and kartevonmorgen.org, therefore we will continue with our first steps in a project so called GoodDB https://git.fairkom.net/fairsync/maps/issues/10#note_23474 and might extend it to activity pub later.

yes, we have been seeing the acceptance of fair sync by NGI0 https://nlnet.nl/project/FairSync/ this is great news.

Dear @natacha and @how

With Fredy @naturzukunft and Markus (Open fair DB) we have got two developers ready to develop the Interface for InCommon and Gogocarto.

Where can we find your specifications?
Can we do a call? All 5 of us?
Fredy @naturzukunft just started our first draft of specifications: https://git.fairkom.net/fairsync/maps/-/tree/master/specification Now we are open for your comments.

Thanks
Helmut

Hi there,
we fond some distributed documentation about the incommon api. is there somebody who has an overview ? I found https://api.incommon.cc but didn’t understand it.
I’ve to try to understand https ://doc.incommon.cc/#introduction and IN COMMON Models

hi @wellemut great to see this.
We should talk asap because there are many things that you are approaching that have already been worked out by IN COMMON Please check https://framagit.org/incommon.cc/

Also we should update you with the latest work done in collaboration with @pukkamustard and openengiadina concerning linkdata specifications that are really important for all of this.

Is there a moment when we can do a group call

Hi @wellemut, thank you for reaching out.

I had a look on @naturzukunft’s API draft and I have three comments from the top of my head:

  1. the licensing information is incompatible: you’re going for Public Domain, but we’re going for AGPL – which means we deliberately want to give an advantage to free software developers over any appropriation available using PD.
  2. Our Place model (see also Resource) is quite different and generally we take a modular approach, e.g., an email is not a property but a relation.
  3. We’ve been discussing with @pukkamustard about moving towards a graph model which would allow us to support a dual approach to data synchronization: interoperability with Linked Data on the one hand, and with peer-to-peer replication on the other hand (e.g., with P2PCollab).

For the latter point, we’ve submitted a joint research proposal to experiment with P2P protocols and graph data models, so this is where we’d like to evolve our current specification, which is currently too much in flux to be presented as a flat document. That said, the source code already provides quite a lot of documentation on the relations between models and the attribute specifications – run rails erd from the source tree to generate the diagrams. I suggest looking at the specs folder in the request-specs branch to get an idea since this is where the most recent reflection appears. I’d be happy to dig into it more, but please have a look at openEngiadina’s Geopub demo[1] to get an idea where we’re headed… @pukkamustard wrote a couple of proposals that you might want to read and comment on as well: have a look at the linked papers on the Encoding for Robust Immutable Storage (ERIS) demo page.


  1. OpenEngiadina and P2PCollab are also supported by NGI0 grants, so we’re happy to collaborate. ↩︎

1 Like

Maybe https://doc.incommon.cc/ can help you read the diagrams.

Maybe we can do an AMA so that you can lift any doubt on our current approach and why we chose what we did…

@how which means “AMA” ?

Ask me Anything :wink:

1 Like

@how
I did not really understand that rails stuff ,-(

I like that linkedData stuff somehow, but have currently no idea how to use it and/or how to define a specification.
For our Minimum Viable Product (MVP) we decided to use a simple openAPI specification with a simple datamodel that is based on the openfairdb API.
This is what all parties understand and are able to implement for now. Our Goal with the MVP is to work and share data in a fast way.

We have stuff like linked data in mind, but not for our MVP. However, it would be nice, to have an openAPI Model, that is mapable on your linked data model.
We will sure not have a common understanding, a defined spec and working plattforms using that spec in the next weeks.

However, we want to understand your thoughts and specification.
For me personally a call is fine, but my personal english communiucation skills are not the best.
I’ve often have trouble to understand the german guys in the video conference :wink:

I would like to use your api like
curl https://api.incommon.cc/resources/bcd7894b-650b-482a-b186-123b0ec33c6c -H ‘Authorization: Token 23d8009f-f2ec-44b6-bcbe-58ad1d3af10b’
and try to understand whats going up. But i didn’t understand that documentation https ://doc.incommon.cc and cannot fin curl commands for other resources like places, locations, events, entities, ?!

Hi @naturzukunft,

Great project you are working on!

Just had a look at the FairSync API Specification. From what I understand that primarily defines a minimal common data model that should be implemented/implementable by all connected maps and platforms. Is that right?

There is a well-defined and widely-used data model to describe these kind of things: schema.org. I think the data model described in the FairSync API specification can be mapped to schema-org.

The advantage of using schema-org as data model is that it is already widely-used (e.g. by GoGoCarto and radar.squat and many more). A lot of existing data does not have to be converted, just fetched.

Furthermore schema-org defines a Linked Data vocabulary and can be directly used over protocols such as ActivityPub. As an example: I was able to import data from GoGoCarto and radar.squat into a ActivityPub client with zero coordination with the respective projects.

I think there would be immense advantage of using an already widely-used data-model as the “common” model.

Connectors from this data model to platforms that do not use this data-model (e.g. Wordpress, Drupal, Android, iCal ,etc.) would be extremely valuable (I believe beyond the initial use-case of of FairSync).

Also am happy to participate in an AMA (auch in Deutsch).

2 Likes

Well, I sent you a few links, including a demo that links to 3 papers who really should read. I understand your MVP is something you have in mind for a “simple” and “fast way” to share data, but you must understand as well that what @pukkamustard proposes is already working in the demo – this time I link to the associated blog article that clarifies the approach of using Linked Data and ActivityPub for content curation – and implementing something too simple may incur more work for everyone in the future. Fast is not necessarily a good approach to interoperability design.

I’m not certain to understand your motivation and goals with regard to merging FairSync and IN COMMON or making them interoperable if you’re not interested in understanding what we have in mind. I can only support @pukkamustard in saying that we’re happy to discuss a common model and indeed our serializations should be compatible with Schema.org definitions to some extent to facilitate interoperability – especially if we’re talking about data synchronization among distinct software approaches.

Maybe we could start with text-form questions so we can prepare a first voice meeting in the coming days. I could for example start from your Place model and compare it with IN COMMON’s and Schema. That could help bootstrap a common understanding of what we’re dealing with. What do you think?

Places and Entities are Resources. You can consume them like any other resources – except you can only write them via their own route (see below[1]). Events are not yet implemented. All routes are defined in config/routes.rb, here is the details[2]:

$ rails routes
         Prefix Verb   URI Pattern                Controller#Action
           root GET    /                          api#index
 api_taxonomies GET    /taxonomies(.:format)      api/v0/taxonomies#index
                POST   /taxonomies(.:format)      api/v0/taxonomies#create
   api_taxonomy GET    /taxonomies/:id(.:format)  api/v0/taxonomies#show
                PATCH  /taxonomies/:id(.:format)  api/v0/taxonomies#update
                PUT    /taxonomies/:id(.:format)  api/v0/taxonomies#update
                DELETE /taxonomies/:id(.:format)  api/v0/taxonomies#destroy
 api_categories GET    /categories(.:format)      api/v0/categories#index
                POST   /categories(.:format)      api/v0/categories#create
   api_category GET    /categories/:id(.:format)  api/v0/categories#show
                PATCH  /categories/:id(.:format)  api/v0/categories#update
                PUT    /categories/:id(.:format)  api/v0/categories#update
                DELETE /categories/:id(.:format)  api/v0/categories#destroy
   api_sections GET    /sections(.:format)        api/v0/sections#index
                POST   /sections(.:format)        api/v0/sections#create
    api_section GET    /sections/:id(.:format)    api/v0/sections#show
                PATCH  /sections/:id(.:format)    api/v0/sections#update
                PUT    /sections/:id(.:format)    api/v0/sections#update
                DELETE /sections/:id(.:format)    api/v0/sections#destroy
api_collections GET    /collections(.:format)     api/v0/collections#index
                POST   /collections(.:format)     api/v0/collections#create
 api_collection GET    /collections/:id(.:format) api/v0/collections#show
                PATCH  /collections/:id(.:format) api/v0/collections#update
                PUT    /collections/:id(.:format) api/v0/collections#update
                DELETE /collections/:id(.:format) api/v0/collections#destroy
     api_agents GET    /agents(.:format)          api/v0/agents#index
                POST   /agents(.:format)          api/v0/agents#create
      api_agent GET    /agents/:id(.:format)      api/v0/agents#show
                PATCH  /agents/:id(.:format)      api/v0/agents#update
                PUT    /agents/:id(.:format)      api/v0/agents#update
                DELETE /agents/:id(.:format)      api/v0/agents#destroy
      api_roles GET    /roles(.:format)           api/v0/roles#index
                POST   /roles(.:format)           api/v0/roles#create
       api_role GET    /roles/:id(.:format)       api/v0/roles#show                                                                                                                                                                                                       [1/253]
                PATCH  /roles/:id(.:format)       api/v0/roles#update
                PUT    /roles/:id(.:format)       api/v0/roles#update
                DELETE /roles/:id(.:format)       api/v0/roles#destroy
      api_users GET    /users(.:format)           api/v0/users#index
                POST   /users(.:format)           api/v0/users#create
       api_user GET    /users/:id(.:format)       api/v0/users#show
                PATCH  /users/:id(.:format)       api/v0/users#update
                PUT    /users/:id(.:format)       api/v0/users#update
                DELETE /users/:id(.:format)       api/v0/users#destroy
       api_maps GET    /maps(.:format)            api/v0/maps#index
                POST   /maps(.:format)            api/v0/maps#create
        api_map GET    /maps/:id(.:format)        api/v0/maps#show
                PATCH  /maps/:id(.:format)        api/v0/maps#update
                PUT    /maps/:id(.:format)        api/v0/maps#update
                DELETE /maps/:id(.:format)        api/v0/maps#destroy
  api_positions GET    /positions(.:format)       api/v0/positions#index
                POST   /positions(.:format)       api/v0/positions#create
   api_position GET    /positions/:id(.:format)   api/v0/positions#show
                PATCH  /positions/:id(.:format)   api/v0/positions#update
                PUT    /positions/:id(.:format)   api/v0/positions#update
                DELETE /positions/:id(.:format)   api/v0/positions#destroy
  api_locations GET    /locations(.:format)       api/v0/locations#index
                POST   /locations(.:format)       api/v0/locations#create
   api_location GET    /locations/:id(.:format)   api/v0/locations#show
                PATCH  /locations/:id(.:format)   api/v0/locations#update
                PUT    /locations/:id(.:format)   api/v0/locations#update
                DELETE /locations/:id(.:format)   api/v0/locations#destroy
  api_addresses GET    /addresses(.:format)       api/v0/addresses#index
                POST   /addresses(.:format)       api/v0/addresses#create
    api_address GET    /addresses/:id(.:format)   api/v0/addresses#show
                PATCH  /addresses/:id(.:format)   api/v0/addresses#update
                PUT    /addresses/:id(.:format)   api/v0/addresses#update
                DELETE /addresses/:id(.:format)   api/v0/addresses#destroy
     api_emails GET    /emails(.:format)          api/v0/emails#index
                POST   /emails(.:format)          api/v0/emails#create
      api_email GET    /emails/:id(.:format)      api/v0/emails#show
                PATCH  /emails/:id(.:format)      api/v0/emails#update
                PUT    /emails/:id(.:format)      api/v0/emails#update
                DELETE /emails/:id(.:format)      api/v0/emails#destroy
      api_links GET    /links(.:format)           api/v0/links#index
                POST   /links(.:format)           api/v0/links#create
       api_link GET    /links/:id(.:format)       api/v0/links#show
                PATCH  /links/:id(.:format)       api/v0/links#update
                PUT    /links/:id(.:format)       api/v0/links#update
                DELETE /links/:id(.:format)       api/v0/links#destroy
     api_phones GET    /phones(.:format)          api/v0/phones#index
                POST   /phones(.:format)          api/v0/phones#create
      api_phone GET    /phones/:id(.:format)      api/v0/phones#show
                PATCH  /phones/:id(.:format)      api/v0/phones#update
                PUT    /phones/:id(.:format)      api/v0/phones#update
                DELETE /phones/:id(.:format)      api/v0/phones#destroy
  api_resources GET    /resources(.:format)       api/v0/resources#index
   api_resource GET    /resources/:id(.:format)   api/v0/resources#show
   api_entities GET    /entities(.:format)        api/v0/entities#index
                POST   /entities(.:format)        api/v0/entities#create
     api_entity GET    /entities/:id(.:format)    api/v0/entities#show
                PATCH  /entities/:id(.:format)    api/v0/entities#update
                PUT    /entities/:id(.:format)    api/v0/entities#update
                DELETE /entities/:id(.:format)    api/v0/entities#destroy
     api_places GET    /places(.:format)          api/v0/places#index
                POST   /places(.:format)          api/v0/places#create
      api_place GET    /places/:id(.:format)      api/v0/places#show
                PATCH  /places/:id(.:format)      api/v0/places#update
                PUT    /places/:id(.:format)      api/v0/places#update
                DELETE /places/:id(.:format)      api/v0/places#destroy
     api_things GET    /things(.:format)          api/v0/things#index
                POST   /things(.:format)          api/v0/things#create
      api_thing GET    /things/:id(.:format)      api/v0/things#show
                PATCH  /things/:id(.:format)      api/v0/things#update
                PUT    /things/:id(.:format)      api/v0/things#update
                DELETE /things/:id(.:format)      api/v0/things#destroy

  1. /resources are for reading any type of resource, while /place, /entity, etc. are for writing. ↩︎

  2. :format is json, it could be xml, etc. but we’re focusing on JSONAPI. ↩︎

1 Like