Most of my work on this is on paper, and some code sketeches. There quite a few non-trivial issues, and hard decisions to be made. Since I have the knack of loosing or recycling the various paper bits lying around, I decided to keep at least some of them in my diary, aka this blog.

initial requirements

convenient syntax and semantics

convenience matters. The syntax should be obvious. The meaning of the syntax should be easy to grasp.

minimal core

the core of the macro processor should be minimal. This will help managing the code quality. The minimal core should be enriched using macros.

retargetable/multi-targets

using one and the same 'surface syntax' to produce different target code. It should be possible to have a one to many code generation. Example application - one definition producing javascript, php and sql code. Alternative example: one definition - three code targets - debug, deployment + a set of autogenerated tests

target and syntax agnostic

no assumptions should be made (at least at top level) about the target or syntax. We should be able to accomodate any theoretically parsable syntax, within reason, of course. Subject to parsing strategy selection, etc...

Without going into theoretical details, some of which are quite alien to me, I'll try to describe some of the challenges that pattern patching rewriting macros might pose for a language like php. After brief explanation what kind of a beast is this, I try to explore some of the finer points, which might cause problems. The intent of this post is to sketch a design and highlight some of the possible issues.

I decided to split the relations api from the more advanced issues realted to the current sqlgen implementation. The main reason for this split is due to the never ending discussions about what is a relation on drupal forums, irc, mailing lists, etc...

The relations api is going to be a simple implementation of a tree and graph indexes, with emebeded SQL. You should be able to do things like - get related nodes; get next, previous, parent, children, sibling nodes.

Long term-wise we should be able to replace the book and taxonomy implementations of the database using this api. I'm intentionally going to make it extremely minimalistic, since I'm really not a supporter of using embdedded sql statements.

I've been reading. Scheming. And I stumpled upon SchemeUnit and SchemeQL: Two Little Languages. I liked the paper. Especially the SchemeQL part. It does state my (intellectual) motivation behind writing relations/sqlgen.

• SQL statements are not checked until execution time. That is error prone.
• SQL statements are not host language statements. They can't be used in the same way or manipulated as the host language statements. Except by using crude text processing, one cannot programatically compose, abstract and refine embedded SQL statements. Code quality and productivity suffers.

I realised that I need to demo, coming soon in flesh, why I'm doing the relations the way I do, and why I have these strange concepts.

Let's start with an example in KIF, the example is lifted from the KIF draft proposed American National Standard (dpANS).

(defrelation number (?x) := (or (real ?x) (complex ?x))) 

What does it mean? Well, is number(x) will be true if x is either a real or complex number. Number is a relation.

Generally speaking we have two ways to define a relation or function - either explictly defining all possible/known cases, or by using some procedure to produce them.

I've updated the code in my relations svn repository.

Some of the changes a very experimental - like the munit.inc - it is implementing bovine tests- simply very basic (beasty), non-complicated, dirty, grazing unit tests. When I was writing the tests I was out of net reach, so no simpletest for me. And now I just like it.

Implemented tests for the majority of sqlgen. There are the beginnings of theme widgets - using drupal forms api for theming, as Adrian suggested earlier. The widget tests need to be refactored. Don't you believe them.

What I need to do next is to figure out how to tie together the data from db and some meta-data so I can generate the different view. The main goal is minimal metadata for maximum impact. We shall see. At the moment nothing more than the table description from db is used.

I uploaded an initial phase runner, drpkg.inc, and an interface to dependency.inc to the bryght svn.

It is a short and self explanatory file.

drpkg_install_exec($packages,$uri) is the thing to be called by installer routines, where $packages is an array of package name and uri should point to a PACKAGES file.

The code assumes that for each package there is a file in modules|themes|theme_engines for each install phase, called package_name.phase, for example relations.install .

Please discuss this in this drupal forum thread

I uploaded the first version of the relations module to the Bryght svn repository. I intend to use it for easier management and brainstorming. When the module is considered alpha quality it will get into drupal's contrib cvs, maybe earlier. I don't want to pollute it earlier than nessesary.

current status

The bulk of the sql query generation has been done. An initial working version of the url parser, implemention a simple relation definition language parser has been done.

the critical things todo

There are a few things, which need to be done in order for the api to be properly usable. The main show-stoppers are the relation nodeapi and themeing of the output. I haven't done anything on the nodeapi side, since I'm not sure about the UI. Needs experimenting. For the themeing, I need to think through themeing based on the field meta-data. Thes goes dangerously into the cck teritory. If anyone has any ideas, please come up and shout.

OK, I was talking, talking, writing, fuming - here is some code. Please comment. I need some feedback. I need help as well - to make this better. The file is attached to this post.

Some explanation is probably needed, or I might end up scoring even higher mad hatter points. My previous scribbles on relations give background on my thoughts on this. So I won't be repeating them. The goal is to produce an api, which can be used to form predicates, essentially queries on various related things. One and the same relation can be used in many ways. For example in a simple triple - (subject,predicate,object) or (left,relation,right), we can have four different questions - the free variables being subject and object. Things get more complicated when you consider relation tuples with a length more than two. This is the main reason to try and create a query generator.

Now some implementation details. The RFS() function. The parameter passing is a bit awkward. It is not elegant, but I had to solve a particular problem there. Consider that the arguments can be generated using a different RFS function. Consider as well that $arg1 and $arg2 can be generated separately or as part of a single expression. There is a need to pass the arguments (signatures essentially) by name, so we can identify them properly, but the order is not guaranteed or even worse, the order and the number of arguments, as well as their shape can't be known in advance. This leads to what I consider a not elegant, but workable solution. I would love to streamline that.

The battle plan for implementing relations in drupal in no particular order.

• framework
  • relations hook
  • composition
  • intersection
  • exclusion
  • relation node type
  • integration with listings api of cck fame
• data model
  • adjacency lists
  • trees
  • term
  • user
• user interface
  • add relation between two things
  • add to tree
  • query builder
    • composition
    • exclusion
    • intersection