What's the best way to parse RDFa, Microdata, etc, store and display info back using a uniform schema/vocabulary (schema.org's for example)

https://stackoverflow.com/questions/7200492

13-01-2021
|

Question

I'm mainly using Ruby to do this but my plan of attack thus far is as follows:

Use the gems rdf,rdf-rdfa, and either rdf-microdata or mida to parse data given any URI. I think it'd be best to map to a uniform schema like schema.org, for example take this yaml file which attempts to describe the conversion between data-vocabulary and opengraph to schema.org:

# Schema X to schema.org conversion
#data-vocabulary
DV:
  name:name
  street-address:streetAddress
  region:addressRegion
  locality:addressLocality
  photo:image
  country-name:addressCountry
  postal-code:postalCode
  tel:telephone
  latitude:latitude
      longitude:longitude
  type:type
#opengraph
OG:
  title:name
  type:type
  image:image
  site_name:site_name
  description:description
  latitude:latitude
  longitude:longitude
  street-address:streetAddress
  locality:addressLocality
  region:addressRegion
  postal-code:postalCode
  country-name:addressCountry
  phone_number:telephone
  email:email

I can then store information found in one format and re-display them with schema.org syntax.

The other part is determining type. I'd model my tables after schema.org and I'd like to know the type of 'Thing' (Thing) a record would be. So if I parse an opengraph type of 'bar', I'd store it is 'BarOrPub' (BarOrPub).

Is there a better way of doing this? Something automated? A solution already out there? Any input appreciated.

EDIT:

So I'm finding that this parses pretty well (where all_tags includes the tags i'm interested in as keys and schema.org equivalent as the value):

RDF::RDFa::Reader.open(url) do |reader|
        reader.each_statement do |statement|
          tag = statement.predicate.to_s.split('/')[-1].split('#')[-1]
          Rails.logger.debug "rdf tag: #{tag}"
          Rails.logger.debug "rdf predicate: #{statement.predicate}"
          if all_tags.keys.include? tag
            Rails.logger.debug "Found mapping for #{statement.predicate} and #{all_tags[tag]}"
            results[all_tags[tag]] = statement.object.to_s.strip
          end
        end
      end

Solution

For the original question, you're on the right track. In fact, we do similar things in the structured-data.org linter. It might be useful for you to check out the GitHub repo. The basic idea is to to format detection and choose the appropriate reader (RDFa, Microdata or whatever). Once read, you'll have a graph. You'll want to run through each statement in the graph and create a new output graph with predicates and types mapped based on your table. So, for instance, if you say dv:name as a predicate in the source graph, you could output schema:name in the output graph.

Determining type will also require a mapping table to come up with the appropriate output type. Note that OGP doesn't actually use rdf:type, so you'll need to find a statement with ogp:type and output an rdf:type along with the mapped class.

Another way to approach the whole thing would be to create an vocabulary with owl:equivalentProperty/equivalentClass assertions and perform OWL entailment to add appropriate triples to the original graph. Ruby's toolset isn't (yet) quite up to this at this point.

OTHER TIPS

Regarding Schema.org mappings, we are collecting relevant links at http://www.w3.org/wiki/WebSchemas. If you produce any new ones, please add them.