dikini.net - scheme

An exercise with continuations, prompts and scheme macros

vlado — Fri, 03 Aug 2007 12:21:39 -0400

control resumed inspired me to take a small exercise.

"Code a minimal approximation of the python yield operator based generators"
Here follow the results I managed to get with help from Matthias Felleisen, via the plt mailing list. You can see the non-edited exchange via the mailing list archives and control and macors. All control&prompt are forms specific to plt scheme's control.ss module. The first attempt looks something like this:

(require (lib "control.ss"))

(define (make-step)
  (define (control-state)
    (yield 1)
    (yield 2)
    (yield 3)
    'finished)
  (define (yield value) (control resume-here (set! control-state resume-here) value))
  (define (generator) (prompt (control-state)))
  generator)

(define step (make-step))

(step) (step)

It does the job. Kind of. For me the interesting bit is how the interplay of prompt and control achieve maintaining the current state of execution before the return and allowing to continue from that interruption. It can be done with call/cc but it looks a helluva a lot more complicated and is far more unreadable.

There is a problem with the code above though. As Matthias notes in his email:

LESSON: never 'test' by running things at the top-level prompt. It IS a prompt. -- Matthias

So comes version 2:

(require (lib "control.ss"))

(define (make-step)
  (define (control-state)
    (yield 1)
    (yield 2)
    (yield 3)
    'finished)
  (define (yield value) 
    (control resume-here 
             (set! control-state 
                   (lambda () (prompt (resume-here)))) 
             value))
  (define (generator) (prompt (control-state)))
  generator)

(define step (make-step))

(equal? '(1 2 3) (list (step) (step) (step)))

control-state is a variable holding a reference to a function with no arguments, doing what the python function would do. yield is a variable holding a reference to a function which does the interrupt magic. generator is a variable holding a reference to a (sugary) function which does the initial step.

The logic of the above algorithm is roughly as follows - mark thy state, do what you need to do, when you encounter control stop, remember your state by wrapping it in a function (resume-here), remember where do you what to continue from ( the set! control-state ...). Notice that the value of control-state is replaced. This is possible since the resume-here continuation is a function and a first class value.

So what next. Boilerplate code is tedious to write. And there is a lot of boilerplate code there. And every function, which needs to follow this recipe, will need to duplicate it.

So scheme macros to the rescue. It is not a trivial problem. A 'generator' function can look something like this:

LISP cycles

vlado — Thu, 02 Aug 2007 05:22:14 -0400

from xkcd

macro processor - initial notes

vlado — Wed, 30 Aug 2006 05:46:40 -0400

Most of my work on this is on paper, and some code sketeches. There quite a few non-trivial issues, and hard decisions to be made. Since I have the knack of loosing or recycling the various paper bits lying around, I decided to keep at least some of them in my diary, aka this blog.

initial requirements

convenient syntax and semantics: convenience matters. The syntax should be obvious. The meaning of the syntax should be easy to grasp.
minimal core: the core of the macro processor should be minimal. This will help managing the code quality. The minimal core should be enriched using macros.
retargetable/multi-targets: using one and the same 'surface syntax' to produce different target code. It should be possible to have a one to many code generation. Example application - one definition producing javascript, php and sql code. Alternative example: one definition - three code targets - debug, deployment + a set of autogenerated tests
target and syntax agnostic: no assumptions should be made (at least at top level) about the target or syntax. We should be able to accomodate any theoretically parsable syntax, within reason, of course. Subject to parsing strategy selection, etc...

macros, higher-order functions, datatypes and design patterns

vlado — Tue, 29 Aug 2006 10:09:21 -0400

While reading, researching and experimenting for my macros project the terms in the title get mixed a lot. I'll try to summarise what is my understanding of a lot of the above. To be fair, most of these words are heavily overloaded with sometimes contradictory and or ovelapping meanings, so this short is only about my personal summary, no pretence for a general study or lit review.

Generic programming intuitevely corresponds to design patterns, or a tool to implement them. There should be a distinction between the functional or algorithmic side of the term and the structural or data (type) genericity.

Poor man's macro programming in php

vlado — Wed, 26 Jul 2006 11:43:44 -0400

Jonnay's post I mentioned before started me thinking - what do you need to have macros in php. What is the closest we can get to that without actually changing anything in php (poor man's version? What minimal sugar does php need to make it comfy? What is the natural syntax for macros in php? I definitely don't know the answers to these questions but let's try:


  macro delay() {
     match { delay( $func, ... ) } : 
       { $varname = $func . random();
         $$varname = array(new promise( $func, ... ), 'evaluate'); 
         return $varname;
       }
     match { $func( ... ) } :
       { return call_user_func($$varname, ... }; }
  }

This looks kind of allright, phpish. It has problems, but the above dream code demonstrates the idea enough - match the left-hand-side code and substitute it with the right hand side. The difference from C macros is that this macro is a program fragment/function/..., the result of which is substituted in the AST, as opposed to simple string pattern matching + substitution

Googles Summer of yawn.

vlado — Wed, 26 Jul 2006 09:07:08 -0400

Googles Summer of yawn. - Google's "Summer of Code" has started and they have announced a few PHP projects. Some of them look interesting and useful. However, I am going to pick apart one in particular, that to be honest, looks neither interesting or useful: The PHP Macro Preprocessor.
....

Why does the PHP Preprocessor need to be stuck in a world of #ifdefs and #includes? Instead of blindly copying what the C preprocessor does, why not focus on the languages strengths and deficiencies? The object system of PHP needs to be looked at, and see if it can be improved upon through a method of code transformation.

[sacrificial rabbit]

I sooo totally agree with Jonnay on this. In fact it is simply like reading my own thoughts, just put it better english, , ok, nearly. But what to do about it? I don't really know, I'm afraid I chicken out from digging into the guts of php. When I last looked there it was really scary.

LISP-like programming in php

vlado — Thu, 22 Jun 2006 09:12:04 -0400

My obsession with things schemish and lispish and having to use php on a daily basis leads to the usual comparison and how would I do this in ... style questions. So here comes another go at how would I program lisp style in php. How much is possible? What is possible? Mind you not everything is really useful or the best way to do it. It is just an intellectual excersise, a study on using some of the php'f features to program in not a typical php (imperative) style.

the compulsory conses, car, cdr in php

Well for the benefit of non-lispers I will use head and tail instead of car and cdr respectively.

Lisp without Parenthesis

vlado — Thu, 04 May 2006 08:11:15 -0400

Well itâ€™s impossible to remove all of the parentheses in Lisp, ..... One alternative to parentheses is being sensitive to white space, in a way that is reminiscent of Python. Lisp without Parenthesis

A good post, which goes along the lines of what I'm playing with as well. He (Peter Schombert) suggests using \ as line continuation mark, which is in line with tradition. I would probably prefer it in the end, as opposed to the beginning of a line, but that's sugar.

I wonder, how hard it would be to add such a language extention to mzscheme.

Me being a noobie, I haven't had a clue that there is an SRFI, which implements that already.

a sample testsuite for the (any/type) base types

vlado — Tue, 14 Feb 2006 08:25:32 -0500

Here comes a sample test suite (it is too long, so only the beginning features in here) for the any/types, I described earlier. I like the clenliness of the schemeunit library. It features a rich set of assertions, simple but powerful grouping and both text and gui result interfaces. In order to use it you need plt scheme. Then you can grab the unit test suite from planet: (require (planet "test.ss" ("schematics" "schemeunit.plt" 1 2)))

The code here shows testing only the any/text type and all relevant auxilary functions, using this datatatype. Looking at the code, it could be turned into a macro, so we could generate type tests automatically, rather than writing this long code..

scheming anything/anytype - creating base types experiment

vlado — Tue, 14 Feb 2006 07:18:16 -0500

This is an experiment to some 'atomic' types in scheme. It was tested only in plt scheme.

The basic idea is that we need some 'atomic' types, from wich to construct everything else in a web cms - users, content nodes, etc... The base field types defined here are - text, stext (short text), date, integer and float.Each field type is a closure, in which some properties, getters, setters and type-checkers are defined.

To define a new field type use the make/any/type macro. In the second part of this code snippet you will find a series of type definitions and helper functions. I quite like how this looks in scheme. I'm surprised actually. This ended up as a very cute OO-like code.

still scheming away

vlado — Fri, 03 Feb 2006 08:45:56 -0500

In scheming_away I started exploring things and stopped somewhere in the middle. Strated with defining things as structures, then as a tree structure. Now comes a third attempt at the same height. Thank god I'm not competing.

Take 3, or coming back to structures


;a tuple is a structure with an id and a value
(define-struct tuple (id value))

;tuple + -> list of tuples
(define make-thing (lambda( head . rest)
  (map (lambda(x)
          (make-tuple 
            (car x) 
            (cdr x))) 
          (cons head rest))))

;just print the bleeding thingread more



scheming away
vlado — Wed, 01 Feb 2006 04:37:49 -0500
In yesterday's write-up I started touching scheme. It is part of my attempt to study the language. It might be a bit ambitious as a first "real" problem, though. So until I reach a solution for that, I will try and solve a few randomly selected different problems. Most of them will be something I already know how to say in other languages, the aim for most of them should be a cleaner solution. In the end I must prove to myself that it is worth learning scheme.
Today's problem
Things or the theory of anything

There were a lot of discussions going in the Drupal development circles around what is a node, what is a user, is a user a node, is a taxonomy term a node, etc... In Amsterdam a few of us were really impressed my anymeta by mediamatic, where everything is a thing.
read more


scheming templates
vlado — Tue, 31 Jan 2006 06:09:53 -0500
I'm intrigued. You might have noticed that I like Lisp. Itellectually, that is. I don't really speak it. I finally decided to dive and try and learn it. On the surface it is not that hard - the language is extremely simple, essentially it has brackets and white space as separators, a couple of special forms - something like built in functions or language constructs in other language terminology, and a few really cool features like quote, unquote, quasiquote, (precious special forms actually), oh, yes, it has an eval.
In order to go forward with the learning I shall try and implement a simple web-design friendly template for xml applications. The idea is similar to TAL, or taglibs, etc...
read more


Some ways to use saved state with closures in php
vlado — Wed, 25 Jan 2006 05:23:50 -0500
In a previous short post Idescribe a way to emulate closures in php. Using that technique execution environment, otherwise known as a call stack can be saved for future use. This can be put to good use. A couple of patterns or programming techniques could be useful in practice.
State pattern
A closure represents a state => implemenation of a state pattern. This is a bit rich. Usually in OO programming the state pattern is implemented by encapsulating different protocols, for denoted states. This is simple to implement by substituting your protocol specification with a different name. Drupal hooks are a good example implementation of this. The following is a simplisting example implementation of a state pattern based on closures. The last argument of the deduced function is the closure itself.

class function_object {
  var $scope;
  function __get($name) {
    return $this->$scope->$name;
  }
  function __call($name,$args) {
    $func =$this->$state."_$name";
    $args[]=$this;
    return call_user_func_args($func,$args);
  }
  function __constructor($scope) {
    $this->scope=$scope;
  }
  function __clone() {
    $this->scope = clone($this->scope);
  }
  function state($state) {
    $this->state=$state;
  }
  function call() {
    //the function body - do some work
  }
}
To save the state for future use, for example as part of an undo cycle:

  $saved = clone($closure);
Call/cc or call with current continuation in php
Closure effectively represent the future of computation. This means that we can say what we want to do, by where we are. Since php doesn't implement any call stack optimisation, the call/cc style of programming makes sense in a limited number of cases. To use this style of programming we need to redefine our functions to use the continuation



 f(x,c) : c(f,x);

(var, continuation) -> continuation



and in php


 //the function closure call method

 ...

 function call($x, $continuation) {

   $this->scope=$continuation; // emulate c(f,x)

   ...

   do whatever you want here

 }

 ...



OK, this is rather crude. In reality this is very limited - it is assumed that the function is defined in a 'global' scope, so we are putting it in some exhisting stack. It will be cleaner if the continuation implements the stack assignment, rather than rely on the function.
read more


Emulating closures in PHP
vlado — Tue, 24 Jan 2006 04:03:26 -0500
In programming languages, a closure is an abstraction that combines a function and a special lexical environment bound to that function (scope). The variables in the lexical environment are designed to retain state information between function calls. Unlike garden-variety functions which retain no memory of what happened in previous calls, closures are capable of storing information across function calls.closulres in computer science (wikipedia)
PHP doesn't have closures as an element of the language. What can we do to emulate it?
We need to be able to encapsulate in one entity scope, state and execution. The most obvious candidate will be a PHP object.

With the introduction of the __get(), __set() and __call() magic functions for PHP classes, something the manual calls overloading, PHP opens the gate to transparently enhance a class to add variables and methods to an object's body. We can abuse the __get() to implemet nested scopes. The state is preserved as part of the object body. The default function (call()) is a protocol to add a uniformity of execution between different function objects.

class function_object {
  var $scope;
  function __get($name) {
    return $this->$scope->$name;
  }
  function __constructor($scope) {
    $this->scope=$scope;
  }
  function call() {
    //the function body - do some work
  }
}


If all $scope objects are function objects we get a closure emulation in PHP. This opens the doors to such fancy ideas as namespaces, modules et.al. This technique opens the possibilities for functional style programming techniques and idioms as delayed evaluation, continuation, ...
Update: I've added closures to my design patterns collection
read more


Learning lessons from Lisp  or patterns and languages in PHP
vlado — Thu, 19 Jan 2006 09:48:26 -0500
In Lisp and its dialects everything is a first class language construct, that is it can be evaluated and changed from within the language. This gives a few very powerful abstractions, which help in constructing short but still readable programs. There are a few very powerful patterns coming from this single concept. 
A function is a first class variable. It can be assigned to.
This is directly applicable in PHP. Since the php symbol table is a big hash (roughly), you can access the function by it's name - for example: 

$func = 'a_function';
$var = $func();

A program is a data structure, a list, so you can manipulate it using the language
read more


A pause for thought and some cut backs
vlado — Wed, 11 Jan 2006 06:05:00 -0500
I've been reading. Scheming. And I stumpled upon SchemeUnit and SchemeQL: Two Little Languages. I liked the paper. Especially the SchemeQL part. It does state my (intellectual) motivation behind writing relations/sqlgen.

SQL statements are not checked until execution time. That is error prone.
SQL statements are not host language statements. They can't be used in the same way or manipulated as the host language statements. Except by using crude text processing, one cannot programatically compose, abstract and refine embedded SQL statements. Code quality and productivity suffers.
read more