Boldly Open

[/xml]

This just sounds right: pipeline steps are listed sequentially, as they should be, and everyone now understands who’s first and who’s next. But, heck, this is a domain-specific language by all means, moreover written in XML. No way this stuff can be reused in different context, and the “strong typing” nature of the Cocoon pipeline (where everything starts with a Generator and ends with a Serializer, assuming not only that the whole world will talk XML but actually that the whole world will talk SAX) makes things even more difficult.

Finally, consider what the Unix genius have brought us:

[code]
$ grep index.html access.log | awk ‘{ print $1 }’ | sort | uniq | wc -l
[/code]

I think there’s no better comment for the above solution than this:

“A designer knows he has achieved perfection not when there is nothing left to add, but when there is nothing left to take away.” (Antoine de Saint-Exupery)

Expressing the pipeline concept with just one character (the | sign) is a clear indicator of what could be achieved when thinking about simplicity: the concept is powerful, yet the user space view of it is as simple as it can get (admittedly, a bit opaque but it doesn’t take much to get used).

So, what do the above snippets bring to us? In this quest for a simple pipeline API we learn that simplicity is key and that the principle of least surprise suggests that the pipeline declaration should happen at once and in an ordered way. Sticking to Java, this leaves us with something like

[java]
pipeline.setupPipeline(List components);
[/java]

or (uglier, but sometimes just effective enough):

[java]
pipeline.setupPipeline(PipelineComponent[] components);
[/java]

Actually I’d much rather see the setup happening during the construction phase, but for the sake of interface design I’ll leave the convenience method for now. This means that our Pipeline interface becomes something like:

[java]
interface Pipeline extends PipelineComponent {

void setupPipeline(List components);

void start();

}
[/java]

Easy and effective: whoever knows the pipeline concept should be able to grasp how this API works in minutes. We need to talk about the PipelineComponent interface though, and this is related to the next wild idea: pipeline machinery.

(warning: shaky ground ahead, this is the part which needs *much* more thinking)

In the OO world things aren’t quite as simple as in a CLI environment, where all you have are basic pipeline contracts such as “whatever byte streams comes from the left side is pumped to the right side”. We have objects here, and I don’t want this generic pipeline API to be strongly typed as in being able to work just with - say - SAX events or other XML gibberish. What I want is an API which is able to work with many formats in a way that’s transparent to the user: this means that the various pipeline stages should be able to express their contracts in terms of required input and output format. It’s up to the pipeline machinery providing adapters and bridges so that, say, a pipeline component working with SAX events might be able to cooperate with another component working with streams. This could be accomplished either through some kind of PipelineDescriptor, with annotations or just through different interfaces: whatever keeps things simple, makes me happy.

Another nice solution comes (again!) from a chat with Sylvain reminding me of the IAdaptable approach in Eclipse. This solution fits like hand in glove with a world of interchangeable and heterogeneous pipeline component stages, even though I have a few concerns thinking about the added complexity for pipeline component writers in implementing an Adaptable strategy: if the first and foremost objective of this API is simplicity, then writing components should be as easy as possible.

Anyway, the final outcome of all this would be something like this during the pipeline assembly phase:

[java]
PipelineComponent current;
PipelineComponent next;

if (next.accepts(current)) {

current.setNext(next);

} else {

PipelineComponent adapter = next.getAdapter(current.class);

current.setNext(adapter);
adapter.setNext(next);

}
[/java]

With this mechanism in place, in theory, the pipeline is much more versatile: sticking to the XML world it would be possible to build pipelines using whatever mix of SAX, DOM, StAX, AXIOM and YouNameWhat. Moreover, it would be easy enough to provide adapters to the stream world, tees and nested pipelines (there is a reason for Pipeline to extend PipelineComponent after all).

Of course I expect the pipeline machinery to do much more than just adaptation: caching, logging, monitoring and management are vital to the pipeline deployer. But the real point of this effort is, once again, simplicity. Once I’m able to do this:

[java]
// Get the default pipeline implementation
Pipeline pipeline = PipelineFactory.getPipeline();

// Set up the pipeline with an array of PipelineComponents
pipeline.setupPipeline({reader, transform1, transform2, streamAdapter });

// grab the InputStream from the latest component
InputStream result = streamAdapter.getInputStream();

// start processing
pipeline.start();

// enjoy results
is.read()…
[/java]

or this:

[java]
// This time we use SAX events straight away
pipeline.setupPipeline({reader, transform1, transform2 });

// connect to the pipeline result
transform2.setContentHandler(myContentHandler);

// start processing and handle events coming in
pipeline.start();
[/java]

or, why not:

[java]
pipeline.setupPipeline({reader, transform1, transform2 });

anotherPipeline.setupPipeline({something, pipeline, somethingElse});

anotherPipeline.start();
[/java]

Then I could do this from within Cocoon:

[javascript]
function handlePage() {

var pipeline = cocoon.newPipeline({ file(”something.xml”, xslt(”foo.xsl”), forms(), i18n() });

cocoon.sendPipelineAndWait(pipeline);
}
[/javascript]

but also, when Cocoon is not an option:

[xml]
< %@ taglib uri="http://jakarta.apache.org/taglibs/pipeline" prefix="pipeline" %>
…