Writing a DSL with Python

[+] ramanujan|16 years ago|reply

I would recommend pyparsing for this. There was an excellent article on it in Python Magazine a few months ago.

Some links:

1. O'Reilly mini-book on pyparsing (probably best single ref) http://oreilly.com/catalog/9780596514235/

2. Quick example of pyparsing http://eikke.com/pyparsing-introduction-bnf-to-code/

3. Main pyparsing page (i know it's not very aesthetically pleasing. The code is much better, I promise) http://pyparsing.wikispaces.com/

4. Links to more on pyparsing here http://pyparsing.wikispaces.com/Publications

Python Magazine is excellent, by the way. Definitely worth the purchase price as they include a zip file with code. (Mandatory disclaimer: I have no stake in it at all. Just found it very useful, that's all).

[+] extension|16 years ago|reply

Note that there is a world of difference between an in-language DSL and one with its own syntax that must be parsed, so much difference that they really should have different names. The former can effortlessly exchange data with and enjoy all the features of the host language. I'm convinced that the latter will one day be an artifact of less enlightened times.

[+] tomlu709|16 years ago|reply

By convention the in-language form is called an internal DSL, the other an external ditto.

[+] spahl|16 years ago|reply

A very interesting DSL for network packet manipulation is Scapy(http://www.secdev.org/projects/scapy/).

It uses python itself as a DSL to make things like packet forging easy:

  >>> Ether()/IP()/TCP()/"GET / HTTP/1.0\r\n\r\n"
  <Ether type=0x800 |IP frag=0 proto=TCP |<TCP |<Raw load='GET / HTTP/1.0\r\n\r\n' |>>>>

Here the / operator was overloaded to stack layers.

[+] hedgehog|16 years ago|reply

Wow. If you ever wanted curly braces or multi-line lambdas in Python this looks like the way to do it.

[+] pavelludiq|16 years ago|reply

What exactly would people do with multi-line lambdas? I would definitely refactor any lambda with statements or more than one expression in it, give it a name, or use a list comprehension, so I've never seen a problem with them. Can anybody give me an example of a piece of code they think would look better if python had multi-line lambdas?

21 comments