.. include:: /std/QuickNote-tip.irst .. _Sieve_in_rPython: =============================== QuickNote: The Sieve in RPython =============================== .. post:: 2023/9/14 :category: RPython, Experiment, rough, QuickNote :tags: Castle, WorkshopTools Before implementing an :ref:`RPython backend `, a short study of `RPython `__ is made, including implementing :ref:`Castle-TheSieve` in RPython. It’s like a hand-compiled RPython variant, following the approach as described in the :ref:`Jinja rendering study `), for the :ref:`CC2CPy backend `. Writing RPython isn’t too complicated for someone with a C-background. But making the code pass the RPython compiler can be tricky, however. Both, as just added statements can interfere with existing code -- when that isn’t “static enough”, new code can trigger compile errors in old, tested code! And, because the RPython compiler isn’t that great in giving helpful info. |BR| It feels like old-style debugging: *try-and-error*, remove-and-enable-lines, one-at-a-time until one understands and it becomes easy to fix... This blog is both to share some RPython experience and to study the patterns that are needed to generate RPython code to implement the Castle-Compiler. Code **** Below we show only a part of the (manually written) code. It roughly follows the “handCompiled-C” version (see :ref:`CC2Cpy`). Again, mostly the structure to get the (Castle) Components working, like sending events between Elements is presented. |BR| All other code is quite straightforward, and can be found in the `code-archive `__ For this study, I had two objectives: (a) make it (functional) work in RPython, and (b) find the patterns to automate. So, do not expect greatly crafted (maintainable) code. Definitions ============ In Castle, you use Events, Protocols and Components (to name a few), all will result in some data structures [#dataclasses]_ that define them: build-in classes that are (statically) instantiated. |BR| This part is easy to automate. .. include:: ./RPython-definitions.irst Data structures =============== Each *(Castle)* component results in a (*RPython*) class containing all code for that component. As (R)Python classes are also namespaces, the names of (e.g.) the even-handlers become shorter; there is no need to put the component-name in the method-name. But in general, we follow the structure as in the :ref:`CC2Cpy` approach. Components ---------- A (Castle) Component becomes a subclass of ``CC_B_Component``, an (abstract) base class providing (currently mostly) debugging support. Subclasses should override the ``_debug_attr_()`` method, which returns *(but not print)* a string with ``name=value`` pairs for debug-printing. The initialiser is split in ``__init__()`` and ``_castle_init()``; the latter implements (generates code for) the Castle-init function, and is called by ``__init__()`` [#init]_. Both initialisers will have the parameters to the class, and/or the elements. A default value (Nil) is set in ``__init__``, whereas ``_castle_init`` can set it -- as specified in the castle-code. See the `SieveClass.py` code for an example (not shown). |BR| Static variables in Components --which are defined in e.g. EventHandlers-- are also handled in ``__init__``. As they have a scope of one function, their name is prefixed with that function-name. You can find an example in the `GeneratorClass.py` (not shown). Roughly each protocol-event for each port (aka an EventHandler) results in a (R)Python method, as do data- (and other) handlers, and internal functions. DispatchTables -------------- Every (input) port can trigger several EventHandlers (or data-/stream-/... handlers), as an (event) protocol can have multiple events. Therefore, a component has an (event) DispatchTable per port [#activePort]_. Every DispatchTable is an *array* (python: List) with the same size as the number of events in the corresponding Protocol (which is static at compile-time) -- including the inherited events [#arbitrary]_. .. include:: ./sub_EH-sidebar.irst Pins ~~~~ A (Castle) component can have sub-components. They can be interconnected using the typical syntax and (RPython) structure. |BR| But a component-element can also receive and send messages to its sub-elements. The (SieveDemo) Main component uses this to handle a newly *found prime* (by the ``Finder``\’s output port ``found``). In Castle-code the EventHandler names both the (sub)element and its port (instead of only a port). See the sidebar for an example. Such an “incoming event” of a sub-element needs a DispatchTable also, we call them “pin(s)”. |BR| Note: No extra DispatchTables are needed to send an event to a sub-element; the normal in-port of that component (aka it DispatchTable) handles that. A bit of extra gear is needed to fire that event, however (see below). Examples --------- .. include:: ./RPython-generated.irst Code fragments ============== To connect two components (via ports), and to send events (or other protocols) over them, some code is needed. That is studied below. Connecting ---------- Castle supports multiple :ref:`Machineries`, with all will generate a bit different code. Here we use the :ref:`Machinery-DirectCall` one [#differ_a_bit]_. We use (one of the ) same examples as in :ref:`QN_Connect` (see: :ref:`QN_Connect_Main__init`). |BR| The Castle Code: .. code-block:: ReasonML .generator.outlet = .finder.newPrime; This will result in two lines of RPython. They are very similar to the C-backend: the `outlet` field holds the other component (element) and the handlers denote the DispatchTable of the incoming port. .. code-block:: python self.generator.outlet.connection = self.finder self.generator.outlet.handlers = FinderClass.cc_S_Finder_newPrime Sending ------- Again, we use an :ref:`earlier example ` (in :ref:`QN_EventMachinery`). The Generator is sending numbers to the sieves .. code-block:: ReasonML self.outlet.input(i); // Part of Generator:: StartSieve.runTo(max) on self.controll This translates to a few lines of Python. |BR| The manual crafted version adds some print/debug and assert statements. It also handles the case where a ``None`` is set in a DispatchTable. See the real code for the details. Here you see the essence. .. code-block:: python outport = self.outlet receiver = outport.connection handlers = outport.handlers handler = handlers[SimpleSieve.CC_P_SimpleSieve_input] handler(receiver, i) Or simply: .. code-block:: python outport = self.outlet index = SimpleSieve.CC_P_SimpleSieve_input handler = outport.handlers[index] handler(outport.connection, i) .. Note:: This code assumes there is always a valid method in all entries of the DispatchTable. See the `generic error function`_ footnote. Trigger sub-elements -------------------- To send an event to an (in)-port of a (Castle) sub-element, we need to select the correct (RPython) DispatchTable, index it with the event number, and call that method, passing the sub-element’s (RPython) instance -- and parameters. .. code-block:: python GeneratorClass.cc_S_Generator_controll[StartSieve.CC_P_StartSieve_runTo](self.generator, max) .. warning:: The Castle Code for this feature isn’t 100% clear. Several variants of the (Castle) Sieve do it differently, wrong or simply lack that line [#CastleBugs]_. ------------------------------------------------ .. seealso:: * https://osdn.net/users/albertmietus/pf/TryOut_PyPy-and-Rpython (SLOW!) |BR| My collection of small rpython demos, and typical Python-code that is not RPython .. * https://osdn.net/users/albertmietus/pf/CC_Sieve_in_rPython (SLOW) |BR| The code used for in this study. .. rubric:: Footnotes .. [#dataclasses] To implement them `dataclasses `__ would be natural. They are, however, introduced in *Python-3.7*. As RPython is a Python-2.7 subset, they are not available. .. [#init] Possibly, it would be better to have 3 init functions. Now the (generated) `__init__`, calls both the `__init__` of the base-component/class and its own `_castle_init`. |BR| This results that all Caste-init code is called, mixed with build-in/generated init code. For now, this approach is used. Remember: the goal in generated code; with different demands on maintainability. .. [#activePort] We could implement this by making the ports “active”; an RPython class with implements a method for each event -- then we use the DispatchTable of Python itself. However, that port method needs to call a method in the ComponentClass -- whose name is not statically determined. |BR| In (normal) Python, this is possible using introspection. That feature is however not allowed in RPython. As all code is generated, it is still possible. We could generate a port method that calls a component method, as we know the name of the latter during (Caste) compile-time. But is that better? |BR| At the moment, we believe a table approach, doing the same thing directly is better! .. [#arbitrary] In this example, we use some arbitrary number (like 6, 7) of “dummy” inherited events for all Protocols. Those *not existing* inherited ones get a simple ``None`` in the table. In real code, this should not happen. Then (a) inherited events are copied down, and (b) empty slots will contain the _`generic error function` [#RT]_. .. [#RT] The runtime becomes simpler (and faster) by using an error function, then using None, and checking for None on every call. For this experiment, both are not needed. .. [#differ_a_bit] As we have seen in :ref:`Machinery-DirectCall`, the code for other machinery differ, but typically not too much. .. [#CastleBugs] During this study, we found several (small) bugs in the Castle Code: typos, wrong names, and (like this) lacking lines. |BR| Probably, that is the result of writing code without a compile/syntax-checker (and so, no options for UnitTesting and/or practising TDD)