“Shed Skin is an experimental Python-to-C++ compiler. It accepts pure Python programs, and generates optimized C++ code. This means that, in combination with a C++ compiler, it allows for translation of Python programs into highly efficient machine language. For a set of 16 non-trivial test programs, measurements show a typical speedup of 2-40 over Psyco, about 12 on average, and 2-220 over CPython, about 45 on average. Shed Skin also outputs annotated source code.”
Is that % speedup, or x speedup? If the average Python program now runs in 1/12th the time it takes in the interpreter, that’s quite an improvement.
I think it’s 1/12th, to bad a “average” python program wont work with it.
right, it (currently) doesn’t work for arbitrary programs. I am not building a ‘magic’ compiler, but something that is mostly useful for developing computationally intensive code. btw, it is of course possible to only compile parts of your program, so you can get very good performance for that part (much better than using Psyco), while still being able to use the full Python in the rest of your program.
Firstly, it is great to have information directly from the author of such an innovative tool.
Secondly I am in the process of writing a translator/ cross compiler in Python. After the pain of Lex + Yacc; the joy of YAPPS was a blessed relief. Will take a look at your work with much interest.
Speed of development with Python is phenomenal compared with C++ or Java; however the pain of recompiling the interpreter to gain additional libraries makes it awkward at times for most folk.
Good luck with progressing your project; it sounds good.
Interesting indeed. Ability to speed up hot parts of a program without rewriting it in C(++) is really, really valuable.
First I’ve thought looking at the source tarball is why isn’t it implemeted in python (performance may somwaht suffer but development might be hell lot easier). Not that i’m criticizing this tool as it seems to be a great one if it achieves production quality (maybe it already has ? still haven’t run it on my linux box but I’m going to try it now
. I’m just curious of your opinion about pros and cons of C++ versus Python implementation.
Regards,
rle
Shed Skin is fully written in Python (ss.py, about 6000 lines). I wouldn’t dare otherwise.. 🙂 it doesn’t compile itself, because it uses several (dynamic) libraries, but I am thinking of compiling the type inference part (which takes 90%+ of the time). all the ?pp files you see are C++ implementations of the builtins and some (parts of) library modules, that are linked with generated code to produce the result.
eh, ok; sorry for dumb questions
oh no, it’s far from production quality (version 0.0.8 kind of indicates this
). it’s really an academic exercise at this moment, partly just to show that it is possible. I am hoping other people will contribute parts, bug reports etc. so that in many months it might work for arbitrary ‘largish’ Python programs (of course, respecting the limitations imposed by using SS).
hi, I’m the author of Shed Skin. sure, we are talking about ‘x speedup’ here 🙂 for example, the Pygmy ray tracer draws a picture in 1 second now, instead of 1 minute using CPython..
I know http://pypy.org has some information about a Python in Python that implements a backend in C and another backend in LLVM. I’m primarily interested in this as a multiplatform environment so Psyco is no use to me.
I have tried to read their website a few times, but it seems they are trying a lot of things at the same time. Shed Skin is really much simpler (only 6000 lines!), and has one clear goal: to be *really* fast for algorithmic-like code, at the cost of some flexibility. also, I am not sure if PyPy will ever accept *pure* code, without any annotations, and if it will ever generate as fast code as Shed Skin does.
!!!
It looks a good stuff to me…
I hope it will soon leave the alpha to be more usable (don’t know exactly how usable it is?)
It is a piece of work anyway. Making a compiler that translate into a different language is not an easy thing, I can imagine. And one guy only.
My only worry is if it may help to leave an excelent “Free” language as Python is…
!!!
i’ve always wondered why the cpython interpreter hasn’t been rewritten, we’re always hearing how slow it is compared with jython, ironpython, psyco etc.
pypy sounds like going in the wrong direction (implementing the interpreter in python would be slower than c, which is already slow enough) shedskin looks promising though.
i seem to recall that the parrot implementation for python had been abandoned too.
psyco doesn’t run python code, it translates python functions to 386 asm. It’s not exactly fully capable, and it’s limited to one architecture. It’s also incompatible with some code.
CPython is largely fine, and it works. It’s slow, but usually that doesn’t matter.
You pretty much just have to put your more complex code in c.
Ironpython also isn’t finished.
I think speedups are actually soon on the todo list for python. I think they’re planning on reworking things for efficiency in python 3, but maybe that’s a rumor?
@ma_d:
CPython is an interpreter, so you can optimize what you want, but it will never be fast. it will always be one or more order of magnitude slower than compared to a good (JIT) compiler. that said, it would be useful of course if CPython got faster 🙂
The point of pypy, as I understand it, is to implement all of the core Python language in Python so that it is extensible. The bytecode interpreter would still be written in C, if not assembler, so it would be plenty fast. Ideally, it would even be faster than CPython is now, since you could focus all your optimization on the bytecode interpreter, and the language itself would be optimized too.
You’ve misunderstood. The point of PyPy is a reimplementation of all of CPython in pure-Python. The point being that it is much easier to experiment with Python code than C code.
pypy is not an interpreter
One of the most strongest point of python is that it’s dynamicly typed. If you drop that, you loose a significant part of python’s strength.
@wkleunen:
sure, true dynamic typing can be useful at times. but I really almost never use it myself. a _lot_ of serious programs are written in C++, without much need for dynamic typing. the real value of Python, as I see it, is that you can _leave out type declarations_, and the high-level syntax and datatypes. Shed Skin maintains all these advantages.
@srepmub:
Indeed, many staticly typed programs are written in C++. Personally i prefer 2 languages: c++ and python. So when you take out dynamic typing from python, i would personally prefer programming in c++.
My personal opinion would be: if you do it, do it fully, like pypy tries to do. If i really am concerned with speed, i will use a staticly typed language.
@wkleunen:
good luck with all the typing 🙂
seriously, use the tool that suits you best. if you enjoy C++ programming, and being less productive than in Python doesn’t bother you, that’s fine. not everyone enjoys C++, or would like to learn it just to have their code run faster.
about pypy: if it is any good, I don’t think I need to reimplement it.
suppose you could write any C++ program ‘at a higher level’, using a Shed Skin like system, so that it runs at comparable speed to manual C++, would you still be writing manual C++..?
Srepmub,
congrats for your effort. What type of parser and tools are you using? do you use an RD parser or a table-driven one? do you use a yacc-like tool?
@axilmar:
thanks! I use the compiler package from the standard library.
of many such projects, with ever improving C++ quality.
One of the promises of .NET was the ability to write a project in multiple languages without the heartburn and performance hit of FFI’s. That’s never really materialized, and I have to suspect that MS’s apparent indifference to their own languishing creation is a trick to gain mindshare until they’re ready to slap on the royalties and lawsuits.
Maybe someday it will be practical to glue generated C++ files, even in weird dialects, together.
I am very curious to try this out. You’ve picqued my interest.
please read about the limitations on the Shed Skin homepage first. it is not as transparent as, say, Psyco, and you may have to modify your program first. let me know if anything breaks, and I would be happy to try and fix the problem.
Thanks very much. I commend your attitude as well. Wouldn’t it be nice if all developers were as welcoming 🙂
great work, out of curiousity what made you chose c++ as the output? why not C or asm?
because C++ is much closer to Python. for example, the Python list type can be mapped to the C++ STL vector type. then you have namespaces, exceptions etc.. C++ just has so many abstractions, which makes translation much easier. (this is one of the reasons why the compiler is only 6000 lines :-))
fair enough