dikini.net - relations

Cut down relations.inc

vlado — Tue, 07 Feb 2006 10:19:18 -0500

Here comes a very short, cut-down version of relations.inc. It is a very basic api for use in various indexing schemes.

It can be expanded to hook into db_rewrite_sql(), but I refuse to do that for the sake of my remaining sanity.

The basic idea is that modules can/should use this or analogous api to perform all "relation" queries.

There is a HUUGE confusion going on about What is a relation?, but I won't participate in that any more - no time, sorry.

Relations api and SQLGEN

vlado — Mon, 23 Jan 2006 11:52:33 -0500

I decided to split the relations api from the more advanced issues realted to the current sqlgen implementation. The main reason for this split is due to the never ending discussions about what is a relation on drupal forums, irc, mailing lists, etc...

The relations api is going to be a simple implementation of a tree and graph indexes, with emebeded SQL. You should be able to do things like - get related nodes; get next, previous, parent, children, sibling nodes.

Long term-wise we should be able to replace the book and taxonomy implementations of the database using this api. I'm intentionally going to make it extremely minimalistic, since I'm really not a supporter of using embdedded sql statements.

minor improvements

vlado — Thu, 12 Jan 2006 10:42:20 -0500

minor imporvements - Rev 4 - vlado (2 file(s) modified)minor imporvements
[relations svn]

As it says on the box minor improvements. Actually the changes are a mockup of some of my ideas for better/proper database abstraction, and a variation on some of Adrian's model thoughts.


  sqlgen_join(
    // functional form
    sqlgen_table('n'=>'node'),
    //array(ish) form
    array('r'=>'node_revisions'),
    // the join constrains, defines the new relation
    sqlgen_constraints(
      array('node_revisions'=>'may be null'), 
      sqlgen_and(
        sqlgen_eq( arg('node','nid'),arg('node_revisions','nid') ),
        sqlgen_eq( arg('node'=>'vid'),arg('node_revisions','vid'))
      )
    ),
    //result filtering
    sqlgen_cond( sqlgen_eq(arg('node','nid'),$nid)) 
  );

This is the functional form. It provides a good abstraction. It is independant of the underlying structure. In my opinion it has better code readability. Can be thoroughly unit tested.


 $view=array(
    '*'=>'node_revisions',
    '*'=>'node',
    'name'=>'user',
    '#cond'=>array(
      '#op'=>'AND',
      array(
          '#op'=>'eq',
          array('node','nid'),
          array('node_revisions','nid')),
      array('#op'=>'eq',
          array('node','vid'),
          array('node_revisions','vid')),
      array('#op'=>'eq',
          array('node','nid'),
          '#param'=>'node_view_id')
      )
    );

The array form, can easily become a mess. It is faster using arrays, rather than using functions to generate them, but could be daunting.

Both cases provide a close native equivalent to s-forms in php. Although you have eval(), the native php syntax is not as easily manipulateable as in lisp.

Still both forms have a significant advantage over 'embeded SQL'. We can get native php expression of the concepts in the db. We can optimise the queries, based on both data knowledge and database backends. Database independence is obvious. No rewrite_sql hacks. So things like coding a new/custom permission system becomes a breeze. Content type mashups, view/form remixing becomes possible.

As I've said this is just a mockup.

updated the comments in the functional form for readability

A pause for thought and some cut backs

vlado — Wed, 11 Jan 2006 06:05:00 -0500

I've been reading. Scheming. And I stumpled upon SchemeUnit and SchemeQL: Two Little Languages. I liked the paper. Especially the SchemeQL part. It does state my (intellectual) motivation behind writing relations/sqlgen.

SQL statements are not checked until execution time. That is error prone.
SQL statements are not host language statements. They can't be used in the same way or manipulated as the host language statements. Except by using crude text processing, one cannot programatically compose, abstract and refine embedded SQL statements. Code quality and productivity suffers.

Relations, declarative knowledge and inference

vlado — Tue, 20 Dec 2005 04:45:10 -0500

I realised that I need to demo, coming soon in flesh, why I'm doing the relations the way I do, and why I have these strange concepts.

Let's start with an example in KIF, the example is lifted from the KIF draft proposed American National Standard (dpANS).


(defrelation number (?x) := (or (real ?x) (complex ?x)))

What does it mean? Well, is number(x) will be true if x is either a real or complex number. Number is a relation.

Generally speaking we have two ways to define a relation or function - either explictly defining all possible/known cases, or by using some procedure to produce them.

relations update

vlado — Mon, 19 Dec 2005 06:50:21 -0500

I've updated the code in my relations svn repository.

Some of the changes a very experimental - like the munit.inc - it is implementing bovine tests- simply very basic (beasty), non-complicated, dirty, grazing unit tests. When I was writing the tests I was out of net reach, so no simpletest for me. And now I just like it.

Implemented tests for the majority of sqlgen. There are the beginnings of theme widgets - using drupal forms api for theming, as Adrian suggested earlier. The widget tests need to be refactored. Don't you believe them.

What I need to do next is to figure out how to tie together the data from db and some meta-data so I can generate the different view. The main goal is minimal metadata for maximum impact. We shall see. At the moment nothing more than the table description from db is used.

drpkg and relations svn

vlado — Tue, 13 Dec 2005 06:38:42 -0500

The bryght repository seems to be down.

So here are the urls to the drpkg and relations svn repositories maintained at MEC by me. I will be syncing them to bryght regularly.

svn://mec0026.engi.cf.ac.uk/relations
svn://mec0026.engi.cf.ac.uk/drpkg

To checkout the relations code do:
svn co svn://mec0026.engi.cf.ac.uk/relations

To checkout the drpkg code do:
svn co svn://mec0026.engi.cf.ac.uk/drpkg

Update: It is back up, but I'll keep both in sync.
Update: repair

Relations Battle Plan II and first results

vlado — Wed, 30 Nov 2005 05:59:57 -0500

I uploaded the first version of the relations module to the Bryght svn repository. I intend to use it for easier management and brainstorming. When the module is considered alpha quality it will get into drupal's contrib cvs, maybe earlier. I don't want to pollute it earlier than nessesary.

current status

The bulk of the sql query generation has been done. An initial working version of the url parser, implemention a simple relation definition language parser has been done.

the critical things todo

There are a few things, which need to be done in order for the api to be properly usable. The main show-stoppers are the relation nodeapi and themeing of the output. I haven't done anything on the nodeapi side, since I'm not sure about the UI. Needs experimenting. For the themeing, I need to think through themeing based on the field meta-data. Thes goes dangerously into the cck teritory. If anyone has any ideas, please come up and shout.

Relations API - query generation and TODO

vlado — Mon, 21 Nov 2005 03:26:17 -0500

OK, I was talking, talking, writing, fuming - here is some code. Please comment. I need some feedback. I need help as well - to make this better. The file is attached to this post.

Some explanation is probably needed, or I might end up scoring even higher mad hatter points. My previous scribbles on relations give background on my thoughts on this. So I won't be repeating them. The goal is to produce an api, which can be used to form predicates, essentially queries on various related things. One and the same relation can be used in many ways. For example in a simple triple - (subject,predicate,object) or (left,relation,right), we can have four different questions - the free variables being subject and object. Things get more complicated when you consider relation tuples with a length more than two. This is the main reason to try and create a query generator.

Now some implementation details. The RFS() function. The parameter passing is a bit awkward. It is not elegant, but I had to solve a particular problem there. Consider that the arguments can be generated using a different RFS function. Consider as well that $arg1 and $arg2 can be generated separately or as part of a single expression. There is a need to pass the arguments (signatures essentially) by name, so we can identify them properly, but the order is not guaranteed or even worse, the order and the number of arguments, as well as their shape can't be known in advance. This leads to what I consider a not elegant, but workable solution. I would love to streamline that.

Relations in drupal links

vlado — Wed, 16 Nov 2005 07:53:15 -0500

A collection of relations in rupal related links. Add more in comments.

Once again relations, or the need to focus on smaller parts of the big picture. A rant.

vlado — Tue, 15 Nov 2005 03:14:48 -0500

After reading The Need for RDF Linking - Module Proposal, RDF Metadata and this comment I felt the urge to engage in writing this spewage.

The aim of the relations "system" I'm working is to capture the structure of a relation domain. Nothing more. Not to try to interpret the meaning of a relation, not to try and build ontologies, semantic networks, not to store different metadata, etc... Not at all.

You might ask why. Well the main reason is that I think that is not a consern for this particular module. I want it to be able to interpret structures well enough, that there is a way to express out of the box more complex things, that we can do now in drupal. As a side effect, we will end up having a library or API with wich we can express things like this thing is a tree. This thing is a graph. This thing is a set of things. this is an ordered set, and the distance between A and B is x. Things like that.

nodes related to this one by tags

vlado — Wed, 09 Nov 2005 11:25:50 -0500

Nasty title. Never the less, the first hurdle is done. Time to wrap up the code. You can see this working on any post within this website. For example this one - you should notice the "Similar content" block. Links are ordered by number of 'my' tags present. As simple as that.

I had a silly problem. mysql 4.0 doesn't support sub-selects, the way my original implementation was coded. Silly, because I spent hours debugging this, without even thinking to check the version.

end result: mysql 1 - vlado 1

I have my test cases for relations now. So things should be out in the wild soon

Relations, SQL, views, algebra or how to cook that broth

vlado — Thu, 03 Nov 2005 04:34:45 -0500

I've been experimenting, again with these. Apart from some bugs, the things are moving. I'll try to summarise my findings and thoughts on this beast called relations.

throw everything into the water

Let's start with a little bit of maths. It gives me that comfort feeling of being serious if I put down some formulas. I might be shooting myself in the head, which is not that comforting.

So, in our case, a relation is a n-tuple - R(X(1),..,X(n)), where Xi are some sets, possibly ordered, or partially ordered sets. The simplest useful relation would be the R(x) - a single value. The equivalent of an SQL databse table is R(Y(1),...,Y(n)), where Y(i) is a column in that table. The equivalent of a SELECT ... WHERE query in this little math markup journey is F(R(Y(1),...,Y(n))), where F() is a filter function implementing the conditions in the where clause. Similarly a row in the db is R(y1,...,yn), where yi is the value for Y(i) in that row.

Relations battle plan

vlado — Mon, 31 Oct 2005 04:37:57 -0500

The battle plan for implementing relations in drupal in no particular order.

framework
- relations hook
- composition
- intersection
- exclusion
- relation node type
- integration with listings api of cck fame
data model
- adjacency lists
- trees
- term
- user
user interface
- add relation between two things
- add to tree
- query builder
  - composition
  - exclusion
  - intersection

Relation modules

vlado — Mon, 31 Oct 2005 04:35:29 -0500

Specifications of a bundle of relations and structures modules. They naturally are tightly connected to SQL, so can be viewed as a kind of SQL query builder.

Goals

explicit relations module
- provide a unified db structure for explicitly related things
- provide a set of functions to get and set relations
- provide nodeapi extension
tree relations module
- provide a db structure for nested set indexes
- provide setter and getters for those
- provide nodeapi extension
related by terms module
- implement the relations api
- provide a db interface
general relations module
- provide a relations api
- provide a set of relation operations - addition, difference
- provide a relations node

Relations and their domain structures

vlado — Sat, 29 Oct 2005 12:53:56 -0400

In this scribble I'll discuss an implementation of different topological structures, which can be used for indexing different "relation systems", for example trees for menus, book structures, etc... graphs for caching links between pages.

We can split the problem into two main subproblems. First, navigation and queries withing a single relation domain. For example local table of contents for a section in a book. Second is higher order or inter-domain queries. For example: given a node, determine the all related nodes, ordering them by their distance from the node, based on taxonomy, their position in a book(s), and their position in the site's "link web". We should be able to implement second and hier order based queries, based on the results of these two basic problems.

Going mad talking about relations

vlado — Wed, 26 Oct 2005 05:31:05 -0400

This is a very dirty and mathematically not precise dialogue between me and me on "general" relations in drupal.

What is a relation

Something taking the form of (object,predicate,subject), or in its general form (objects,predicate,subjects). The predicate can be viewed as a label denoting the relation. In a lot of cases, the predicate of a relation is implied.

Yes, but what does that mean?

Well, with relations we can express things like X is part of Y or x,y,z are Y