dikini.net - programming

He Sells Shell Scripts Not Just to Intersect Sets

vlado — Fri, 07 Mar 2008 06:42:37 -0500

He Sells Shell Scripts to Intersect Sets is a good read. Set and multiset operations from the command line or in .sh. Rather clean, straightforward shell magic.

Science paper of the month, IMHO

vlado — Fri, 10 Aug 2007 09:06:00 -0400

In this paper we will propose the first — to our knowledge — method that finds the global optimum of the TSP by reducing the NP complexity of the problem to N2 complexity. We achieve this by using white light interferometry
(WLI)[14]. From the mathematical point of view we unfortunately haven’t disapproved the commonly believed conjecture that NP 6= P (the Clay mathematics institute set out a price of 1.000.000 US-$ for doing that) but we apparently reduced the complexity by a trick, namely replacing operations by photons (and the physics of interference).

In section 2 we introduce the method as a gedankenexperiment[thought experiment in friendly german]....
http://www.opticsexpress.org/abstract.cfm?id=140598

Just cool. It reminds me of the often forgotten analogue computers. They are very rarely used in practice these days, although they could be used to solve predefined probles fast, furious and sparkling.

With the introduction of uncertainty and loops you could get some very interesting results. Just think about the offspring of a theremin and a miniMoog on acid. Raving BEAM bots? Pacifist fighter drones?

An exercise with continuations, prompts and scheme macros

vlado — Fri, 03 Aug 2007 12:21:39 -0400

control resumed inspired me to take a small exercise.

"Code a minimal approximation of the python yield operator based generators"
Here follow the results I managed to get with help from Matthias Felleisen, via the plt mailing list. You can see the non-edited exchange via the mailing list archives and control and macors. All control&prompt are forms specific to plt scheme's control.ss module. The first attempt looks something like this:

(require (lib "control.ss"))

(define (make-step)
  (define (control-state)
    (yield 1)
    (yield 2)
    (yield 3)
    'finished)
  (define (yield value) (control resume-here (set! control-state resume-here) value))
  (define (generator) (prompt (control-state)))
  generator)

(define step (make-step))

(step) (step)

It does the job. Kind of. For me the interesting bit is how the interplay of prompt and control achieve maintaining the current state of execution before the return and allowing to continue from that interruption. It can be done with call/cc but it looks a helluva a lot more complicated and is far more unreadable.

There is a problem with the code above though. As Matthias notes in his email:

LESSON: never 'test' by running things at the top-level prompt. It IS a prompt. -- Matthias

So comes version 2:

(require (lib "control.ss"))

(define (make-step)
  (define (control-state)
    (yield 1)
    (yield 2)
    (yield 3)
    'finished)
  (define (yield value) 
    (control resume-here 
             (set! control-state 
                   (lambda () (prompt (resume-here)))) 
             value))
  (define (generator) (prompt (control-state)))
  generator)

(define step (make-step))

(equal? '(1 2 3) (list (step) (step) (step)))

control-state is a variable holding a reference to a function with no arguments, doing what the python function would do. yield is a variable holding a reference to a function which does the interrupt magic. generator is a variable holding a reference to a (sugary) function which does the initial step.

The logic of the above algorithm is roughly as follows - mark thy state, do what you need to do, when you encounter control stop, remember your state by wrapping it in a function (resume-here), remember where do you what to continue from ( the set! control-state ...). Notice that the value of control-state is replaced. This is possible since the resume-here continuation is a function and a first class value.

So what next. Boilerplate code is tedious to write. And there is a lot of boilerplate code there. And every function, which needs to follow this recipe, will need to duplicate it.

So scheme macros to the rescue. It is not a trivial problem. A 'generator' function can look something like this:

Drupal, computer science, random bits

vlado — Wed, 13 Jun 2007 04:50:48 -0400

There are quite a few interesting computer science artefacts in Drupal. I'm going to try to highlight a few of them. Sorry, this is unfinished, and might not be finished ever, but I'm just trying to capture a snapshot of confused meandering thoughts.

Very late, dynamic binding

Late binding came to popular life with the advent of object oriented languages. Essentially it means binding of values, for example functions, to names at object creation, as opposed to compile or link time. In drupal, this can happen at any time, more even, it is algorithmic - that is you can change at runtime what is to be executed at a specific control point. The hook system is one of the ways to do it.

Models, records, databases, friends and foes

vlado — Fri, 27 Oct 2006 11:01:12 -0400

the beginnings of a long on DB related pattern discussion.

First sketches

vlado — Fri, 27 Oct 2006 10:57:32 -0400

Ok, Let's recap:

Relational database (SQL) tables are records. We can view them (in our heads) as record type definitions.
JOINS define (some kind of) derived union types.
SQL CROSS JOINS are full cartesian products, i.e. union types where no name folding/aliasing is done by the imaginary type system

How can we model that in php?

Classes (and objects) are records. Arrays are records too. Both are candidates for doing the job. We need to be able to somehow represent this (meta) type information and manipulate it. The aim is to have a natural feeling/looking abstraction of the database in php. It should be flexible enough, for us to modify the relations at runtime, as we need. It should allow us in the long run to have a near optimal speed and not too much complications in the end code. That is a tough cookie.

Problems of active record and friends

vlado — Fri, 27 Oct 2006 06:25:45 -0400

After a lot of silence caused by lot's of work (good, my bills are happy) and a continuous diversion into scheme, haskell, dylan and other interesting languages, I'm back into php speaking land. Funny feeling that. So here comes the beginning of something I've been continuously rubbing my (leftovers of) brains against.

copout This is not php specific, but since it discusses frequent problems occuring in php apps, I've labeled it with as php related as well. Makes it easier maintaining the site you see.

I have a problem with active record as a pattern. It is a logical one. Essentially it turn a (SQL) database row into an object. The class of the object represents the SQL table or view. Of course you can add behaviour to those classes and objects, i.e. activating those records.

All that is nice and dandy, but when you start talking about relations and constructing dynamically queries and corresponding objects, we start hitting the limitations of what I will call from now on the naive active record.

The problem is not trivial at all. It boils down to how to make peace between the host language type system (for example classes) and the SQL's dynamic compound types - the relations expressed as SELECT variants for example. It gets even harder when we decide to reverse the direction, so that we actulally want to update the database. Yuk! As though somebody actually does that!

macro processor - initial notes

vlado — Wed, 30 Aug 2006 05:46:40 -0400

Most of my work on this is on paper, and some code sketeches. There quite a few non-trivial issues, and hard decisions to be made. Since I have the knack of loosing or recycling the various paper bits lying around, I decided to keep at least some of them in my diary, aka this blog.

initial requirements

convenient syntax and semantics: convenience matters. The syntax should be obvious. The meaning of the syntax should be easy to grasp.
minimal core: the core of the macro processor should be minimal. This will help managing the code quality. The minimal core should be enriched using macros.
retargetable/multi-targets: using one and the same 'surface syntax' to produce different target code. It should be possible to have a one to many code generation. Example application - one definition producing javascript, php and sql code. Alternative example: one definition - three code targets - debug, deployment + a set of autogenerated tests
target and syntax agnostic: no assumptions should be made (at least at top level) about the target or syntax. We should be able to accomodate any theoretically parsable syntax, within reason, of course. Subject to parsing strategy selection, etc...

macros, higher-order functions, datatypes and design patterns

vlado — Tue, 29 Aug 2006 10:09:21 -0400

While reading, researching and experimenting for my macros project the terms in the title get mixed a lot. I'll try to summarise what is my understanding of a lot of the above. To be fair, most of these words are heavily overloaded with sometimes contradictory and or ovelapping meanings, so this short is only about my personal summary, no pretence for a general study or lit review.

Generic programming intuitevely corresponds to design patterns, or a tool to implement them. There should be a distinction between the functional or algorithmic side of the term and the structural or data (type) genericity.

Rewriting macros - the peculiar case of php

vlado — Wed, 16 Aug 2006 08:29:03 -0400

Without going into theoretical details, some of which are quite alien to me, I'll try to describe some of the challenges that pattern patching rewriting macros might pose for a language like php. After brief explanation what kind of a beast is this, I try to explore some of the finer points, which might cause problems. The intent of this post is to sketch a design and highlight some of the possible issues.

out of the (php) loop

vlado — Wed, 09 Aug 2006 12:32:03 -0400

inspired by Verity , my urge to display my writing talent and random buzz on the internet

This conversation was (allegedly) overheard in a nursery, which won't be disclosed to avoid litigation

Write a standard php loop on the nursery blackboard

Come on, can't be bothered. Are you joking?
foreach( $array as $key => $value ) echo "$key $value";

Wrong! Don't you ever learn proper language usage

What have I done NOW?

Poor man's macro programming in php (revisited)

vlado — Tue, 01 Aug 2006 04:31:59 -0400

How exactly can you do macros in php now?

Macros have (let's say) two main responsibilities - adding syntactic sugar and abstraction of common patterns. The syntactic sugar bit is something which probably should happen at compile time, at least in a weakly typed language like php. So I'll skip them. The abstraction of computation, well this is wat codified design patterns are. This is what function, classes and the rest of the things we read about and use usually do. To abstract a function in php there are at least two different strategies, probably more, but these two are effective in diametrically different situations. The first one is a combination of the ideas of generic functions (higher order functions) and partial evaluation. In code it can look something like:

Poor man's macro programming in php

vlado — Wed, 26 Jul 2006 11:43:44 -0400

Jonnay's post I mentioned before started me thinking - what do you need to have macros in php. What is the closest we can get to that without actually changing anything in php (poor man's version? What minimal sugar does php need to make it comfy? What is the natural syntax for macros in php? I definitely don't know the answers to these questions but let's try:


  macro delay() {
     match { delay( $func, ... ) } : 
       { $varname = $func . random();
         $$varname = array(new promise( $func, ... ), 'evaluate'); 
         return $varname;
       }
     match { $func( ... ) } :
       { return call_user_func($$varname, ... }; }
  }

This looks kind of allright, phpish. It has problems, but the above dream code demonstrates the idea enough - match the left-hand-side code and substitute it with the right hand side. The difference from C macros is that this macro is a program fragment/function/..., the result of which is substituted in the AST, as opposed to simple string pattern matching + substitution

Updated design pattern collection

vlado — Wed, 26 Jul 2006 06:22:18 -0400

I guess I'm on the down form my design pattern tantrum hit. I updated the collection so it now has about 20 of them, not all can be considered "officially" design patterns, but to be honest I don't really care. I'm very well aware that I ignore the existance of boundaries between design patterns, idioms, techniques, ... That is intentional. I like my rainbow coloured world and I want to keep it that way. If you disagree - just read the small script. My tanrum energy is over and I need to do work work. So the combing over the php design pattern collection will come later. For now enjoy, and have anger fits of your own.

Oh yes. I'm using a particular case of wishful thinking - using any php callable in $callable(...) expressions. The reality is a bit uglier. We need to use either call_user_func($callable,...); call_user_func_array($callable,...) How I wish php could do that out of the box, especially when it actually does it, just not in the short syntax. Anyone happy to hold my hand to do a php patch for that feature? I'm very unfamiliar with the internals, sorry.

Scoped php functions or more ways to abuse php

vlado — Mon, 24 Jul 2006 06:09:03 -0400

Here we go again. In this writeup I'll revisit the ideas of closures, state and closures, partial evaluation and generics.

Basics of 'functions as first class language objects'

Yep. PHP can assign functions to variables. In this sense functions are first class objects. Object and class methods are similar, you can, kind of, assign them to variables and later execute them. What does this change? Well, it improves our ability to abstract common patterns and idioms. There are examples of this in the above writeups as well as in the aspects one and quite a few of the rest of php ones on this site. The basic concept is php callback for example:


call_user_func('a_callback_function');
call_user_func(array('AClass', 'aCallback'));
call_user_func(array($obj, 'aCallback'));

$callback = 'a_callback_function';
$callback();
$callback = array($obj, 'aCallback');
$callback();

What can we use this for? The most obvious and straight forward this is a lightweight state pattern implementation - $a_thing() Will behave differently depending on the value of the $a_thing variable, it's state. Another way to look at php callbacks is as proxies for 'real' functions, class or object methods. Oh, let's not forget is_callable, it's a very useful little number.

Objects and functions

The facility of PHP objects to encapsulate a state and us being able to access this state later can be used to turn them into functions. Let's consider: class aClass{ var $state; function get() { return $state; } function next() { $state=random; } function callbacks() { return array( array($this,'get'), array($this, 'next'));} } $obj = new aClass(); list( $random, $next_random ) = $obj->callbacks(); echo $random(); echo $next_random();

Does Visual Studio Rot the Mind?

vlado — Mon, 03 Jul 2006 11:14:29 -0400

This talk dissects the code generated by Visual Studio; analyzes the appalling programming practices it perpetuates; rhapsodizes about the joys, frustrations, and satisfactions of unassisted coding;
Does visual studio rot the mind

Nice. I like that article, not so much because it slags a Redmond born product, but that it goes to the core - bring back fun, in all it's forms, into programming. Just say no to ugly code.

I really want elegant code back. Unfortunately there is too much of the quick fix hack smartly generated crap bloat.

Partial evaluation and generics in php

vlado — Mon, 03 Jul 2006 09:11:49 -0400

This is a followup of a few of my previous posts (lisp-like programming, aspects and honey, patterns & languages, closures), which describe some programming idioms typical for lisp-like languages.

simple partial evaluation

Here I will add some very short examples for how to do partial evaluation in php. The most straight forward approach would be something like:

aspects and honey in php

vlado — Wed, 14 Jun 2006 11:06:18 -0400

The Drupal way in programming is 60% about hooks 10% about nodeapi and the rest is ingenuity. That's allright, but one fact has always troubled me. And that is - how much effort is spent on discovering which module implements the hook I need. Far enough, the old caching/memoizing trick does the job to ensure that the cpu time is not expensive. Yeah, but still it's outside simplicity might be, just might be implemented in a more elegant fashion.

Drupal hooks implement a nearly Aspect Oriented API. The cross-cut is the point where the hook is invoked. aspect(cross-cut) === hook. The different aspects are declared as moduleName_hook. The disatvantage is that these are indiscriminate. Hooks are always fired. Pros - simplicty, cons - wasted cpu cycles.

I wonder. Is it possible to achieve a similarly simple syntax, but with the added advantage of not firing unnessesary hook calls.

Lisp without Parenthesis

vlado — Thu, 04 May 2006 08:11:15 -0400

Well itâ€™s impossible to remove all of the parentheses in Lisp, ..... One alternative to parentheses is being sensitive to white space, in a way that is reminiscent of Python. Lisp without Parenthesis

A good post, which goes along the lines of what I'm playing with as well. He (Peter Schombert) suggests using \ as line continuation mark, which is in line with tradition. I would probably prefer it in the end, as opposed to the beginning of a line, but that's sugar.

I wonder, how hard it would be to add such a language extention to mzscheme.

Me being a noobie, I haven't had a clue that there is an SRFI, which implements that already.

Summer of Code applications

vlado — Thu, 04 May 2006 04:38:05 -0400

Funny things these summer of code applications. Some of them come a very well thought out projects, others are more like cloudy ideas.

If you want to submit a proposal, and want to be successful, please put a little effort in spec-ing properly. Tangible goals are a very good thing. Good language, not really importnat, just put them in bullets if you fancy it that way.

If unsure, get to #drupal on irc.freenet.org, and ask. Mention SOC and project proposal, and you'll get attention and tlc. That is if you are applying for a drupal project, for others, well don't expect much help in that channel.

SQL whining time

vlado — Wed, 03 May 2006 03:50:37 -0400

I must admit, I'm doing it too. But why oh why, people thing that they must use an sql server, when writing a web application framework. It is beyond me.

Some will say, "Well my/pg/ms/any sql server is fast, it it there, it is ubiquous, flexible and everyone is doing itr, so you can find books about it.

Let's tackle these pros one by one in our context.

Fast

Indeed, it can be. SQL server software is usually designed to work with vast amounts of data, stored on some secondary storage. Relational data can be organised, searched, extracted fast. Sure. But. There is always one. A web application framework doesn't know the exact nature/model of the data. It usually builds a palette of primitive types, then does some magic to join those types per application type. You'll find object-relational mappings, sql synthesis algorithms, others, whend dealing with this problem.

Programmable web

vlado — Fri, 28 Apr 2006 07:30:06 -0400

These unfinished series or articles colloects some of my ideas about the programmeable web. I don't pretend them to be original, exclusive or useful to anyone but me. Although I hope so.

x|s|p expressions mixing

vlado — Fri, 28 Apr 2006 03:16:08 -0400

To continue my random ramblings on the programmable web let's get into the other side. Theming.

These days it is a common place to have a theme engine available in almost any respectable web application. It's the result of the spearation of concerns mantra and the desire to get designers and arty types off your back, give them the freedom to do almost anything they want, while not bothirng yourself (as a developr/admin) with too much unnessesary support requests. Theme engines and ideas how to build them are plenty. They differ a lot. But let's try and dissect what are they doing, or more likely what should they do.

Random thoughts on web application internals

vlado — Thu, 27 Apr 2006 10:16:25 -0400

I've been playing lately with a lot of different ideas. Part of it is to further my own education, part of it is looking for good software design tips. Although there exist quite a few good applications or frameworks, both open source and closed, I can't recommend any as really flexible. By really flexible, I mean easy ways to change its behaviour, how it reacts to the outside world. Most frameworks and applications excell at being themselves, and are rarely excelling at being really good citizens, they are inward looking, as far as their configuration and query is concerned. These are bold statements. They are quite subjective and probably useless. I'll try to explain what I really mean by positive examples.

da code, types of functions

vlado — Mon, 24 Apr 2006 08:33:55 -0400

I've been reviewing, scratching and descratching the idease for the language core, which for the time being I'll be calling da code.

Everything revolves around a few fundamental questions about functions. What is a function? What is a function behaviour? What is the effect of one function's evaluation(behaviour) on the overall ecosystem of a program?

Function composition

In my head there are two fundamental kinds of function composition - sequential and parallel. It is true that they are redundant, since each of them can be expressed by the other, but in an attempt to provide a relatively brief and expressive language, I will consider both. This should help with intuitive application of different computational schematics.

An alternative for multipage forms

vlado — Thu, 23 Mar 2006 02:57:01 -0500

Both yesterday and earlier I was writing about some ideas on how to handle multipage forms in drupal and php in general, while keeping consistent state. OK, here is another variation on the same theme. Code will speak for itself best in this case:
..... try { //this detects the nessesary form step and throws an appropriate exception init_form($request); } catch( FormStep1e $f ) { //do some specific handling } // ... other caught form steps catch( FormStepNe $f ) { //do some specific handling } render_form($form) ....

Advantages of this approach:

Natural program order, even for coding across requests
obvious, switch-like semantics
intuitive and explicit flow tracking

Obviously this can be coded with switch() statements, but in my opinion this is a better approach, since combined with properly serialized state, we are able to significantly shorten the nessesary bootstrap process.

On a further note, the same technique can be used to handle continuations-lite in php. The prerequisite is to be able to somehow save or serialise the runtime environment. Having that, we can do something like:
try { process_request(); } catch( ContExn $cont){ $cont->exec() }
process_request() restores the environment into the ContExn object and throws the exception.

Multi-page forms using form api

vlado — Wed, 22 Mar 2006 02:46:52 -0500

After months of work between hunmonk, myself, and chx, we've finalized some substantial progress on multipart forms. A multipart form is a form that is spread across multiple pages (like a wizard), but doesn't truly "submit" until the final page.
Morbus Iff on the drupal development mailing list

That is a cool functionality. While it shows the capability of the forms api to handle multi-page forms via the #pre_render function, which is good, the approach is only the beginning of multi-page form handling.

Some time ago I was writing about a proposed method for resolving application state across multi-page forms. The sceme applies directly.

Wizardy, druidy, wiz

vlado — Tue, 14 Feb 2006 10:59:44 -0500

For a while I've been looking at some interesting techniques for improved state tracking for web applications. Seaside and plt scheme webserver are using continuations to improve session handling, and have a nice clean code while at it. What does this mean? Continuations are a functional goto. They allow you to capture and save the state of an application, and possibly to return to it at any time. You could think of it as a snapshot of the current state of the application. Well, not exactly, but for the current discussion this will suffice.

a sample testsuite for the (any/type) base types

vlado — Tue, 14 Feb 2006 08:25:32 -0500

Here comes a sample test suite (it is too long, so only the beginning features in here) for the any/types, I described earlier. I like the clenliness of the schemeunit library. It features a rich set of assertions, simple but powerful grouping and both text and gui result interfaces. In order to use it you need plt scheme. Then you can grab the unit test suite from planet: (require (planet "test.ss" ("schematics" "schemeunit.plt" 1 2)))

The code here shows testing only the any/text type and all relevant auxilary functions, using this datatatype. Looking at the code, it could be turned into a macro, so we could generate type tests automatically, rather than writing this long code..

What language I would like to program net applications in?

vlado — Thu, 09 Feb 2006 05:16:20 -0500

What should be the language for the web? Many people will swear it is php. It's best features are immediacy and the fact that it does not look scary, initially at least. Loads of libraries as well. Some will say ruby - not sure about that, never really used it. I - don't know.

Programing the network should be seamless. This means that the hard things should have obvious solutions transparent in the language constructs. For example a web gnome druid like code snippet shouldn't look far harder than:

druid (step1 step2 step3 step4)

The webserver coming with plt is a good example for this style of implementation - here is a snippet: