The biggest blocker to port Zope to Python 3 is RestrictedPython.
What is RestrictedPython?
It is a library used by Zope to restrict Python code at instruction level to a bare minimum of trusted functionality. It parses and filters the code for not allowed constructs (such as open()) and adds wrappers around each access on attributes or items. These wrappers can be used by Zope to enforce access control on objects in the ZODB without requiring manual checks in the code.
Why is RestrictedPython needed?
Zope allows writing Python code in the Zope management interface (ZMI) using a web browser (“through the web” aka TTW). This code is stored in the ZODB. The code is executed on the server. It would be dangerous to allow a user to execute arbitrary code with the rights of the web server process. That’s why the code is filtered through RestrictedPython to make sure this approach is not a complete security hole.
RestrictedPython is used in many places of Zope as part of its security model. An experiment on the Zope Resurrection Sprint showed that it would be really hard to create a Zope version which does not need RestrictedPython thus removing the TTW approach.
What is the problem porting RestrictedPython to Python 3?
RestrictedPython relies on the compiler package of the Python standard library. This package no longer exists in Python 3 because it was poorly documented, unmaintained and out of sync with the compiler Python uses itself. (There are whisperings that it was only kept because of Zope.)
Since Python 2.6 there is a new ast module in the Python standard library which is not a direct replacement for compiler. There is no documentation how to replace compiler by ast.
What is the current status?
Several people already worked on various Plone and Zope sprints and mostly in their spare time on a Python 3 branch of RestrictedPython to find out how this package works and to start porting some of its functionality as a proof of concept. It seems to be possible to use ast as the new base for RestrictedPython. Probably the external API of RestrictedPython could be kept stable. But packages using or extending some of the internals of RestrictedPython might need to be updated as well.
What are the next steps?
Many Zope and Plone packages depend on RestrictedPython directly (like AccessControl or Products.ZCatalog) or indirectly (like Products.PythonScripts, plone.app.event or even plone.app.dexterity).
When RestrictedPython has successfully been tested against these packages porting them can start. There is a nice list of all Plone 5.1 dependencies and their status regarding Python 3.
Our goal is to complete porting RestrictedPython by the end of March 2017. It opens up the possibility guiding Zope into the Python 3 wonderland by the end of 2017. This is ambitious, especially if the work is done in spare time besides the daily customer work. You can help us by either contributing PullRequests via Github or review them.
We are planning two Zope sprints in spring and autumn 2017. Furthermore we are grateful for each and every kind of support.
Are you aware that Python sandboxing is broken by design? https://lwn.net/Articles/574215/The right design is to run Python in a sandbox, not the opposite!
RestrictedPython is not a sandbox. It restricts the Python code before its execution by manipulating the abstract syntax tree instead of trying to make the environment safe for running arbitrary code.
We are aware that this will not result in absolute security but it makes the possible attack surface much smaller.
“RestrictedPython is not a sandbox. It restricts the Python code before its execution (…)” Executing restricted Python code is the definition of a sandbox 🙂 Good luck with that 😉
Nice writeup!
The timeline couldn’t be better for Plone!
The link to Plone 5.1 dependencies should be updated to http://jenkins.plone.org/py3 though, as my blog link (gil.badall.net) is already outdated.
Thanks, I fixed the URL in the blog post.