Parser/Composer library for Python https://fdik.org/pypeg2
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

137 lines
4.0 KiB

  1. #!/usr/bin/python3
  2. """
  3. Parsing sample
  4. To parse we're giving a text to parse and an thing with a grammar. The default
  5. setting includes skipping of whitespace, so we don't need to take care of that.
  6. The comment parameter is set to C style /* comments */
  7. >>> f = parse("int f(int a, long b) { do_this; do_that; }", Function, comment=comment_c)
  8. Because function has a name() in its grammar, we can access this now as an
  9. attribute. With Python 2.7 this gives Symbol(u'f'), with Python 3.2 it gives Symbol('f'):
  10. >>> f.name
  11. Symbol('f')
  12. A Function has an Attribute "parms" in its grammar, which directs to class
  13. Parameters.
  14. >>> f.parms # doctest: +ELLIPSIS
  15. Parameters([(Symbol('a'), Symbol(a[int]) at 0x...), (Symbol('b'), Symbol(b[long]) at 0x...), ])
  16. Because Parameters is a Namespace, we can access its content by name.
  17. >>> f.parms["a"] # doctest: +ELLIPSIS
  18. Symbol(a[int]) at 0x...
  19. Its content are Parameter instances. Parameter has an Attribute "typing".
  20. >>> f.parms["b"].typing
  21. Type('long')
  22. The Instructions of our small sample are just words. Because Function is a
  23. List, we can access them one by one.
  24. >>> f
  25. Function(['do_this', 'do_that'], name=Symbol('f'))
  26. >>> print("f is " + repr(f[0]))
  27. f is 'do_this'
  28. The result can be composed to a text again.
  29. >>> f.append(Instruction("do_something_else"))
  30. >>> print(compose(f))
  31. int f(int a, long b)
  32. {
  33. /* on level 1 */
  34. do_this;
  35. /* on level 1 */
  36. do_that;
  37. /* on level 1 */
  38. do_something_else;
  39. }
  40. <BLANKLINE>
  41. pyPEG contains an XML backend, too:
  42. >>> del f[2]
  43. >>> from pypeg2.xmlast import thing2xml
  44. >>> xml = thing2xml(f, pretty=True)
  45. >>> print(xml.decode())
  46. <Function typing="int" name="f">
  47. <Parameters>
  48. <Parameter typing="int" name="a" />
  49. <Parameter typing="long" name="b" />
  50. </Parameters>
  51. <Instruction>do_this</Instruction>
  52. <Instruction>do_that</Instruction>
  53. </Function>
  54. The XML backend can read XML text and create things:
  55. >>> from pypeg2.xmlast import xml2thing
  56. >>> xml = b'<Function typing="long" name="g"><Parameters><Parameter name="x" typing="int"/></Parameters><Instruction>return</Instruction></Function>'
  57. >>> g = xml2thing(xml, globals())
  58. >>> g.name
  59. Symbol('g')
  60. >>> g.typing
  61. Type('long')
  62. >>> g.parms["x"].typing
  63. Type('int')
  64. >>> print("g[0] is " + repr(g[0]))
  65. g[0] is 'return'
  66. """
  67. from __future__ import unicode_literals, print_function
  68. from pypeg2 import *
  69. # A Symbol can be an arbitrary word or one word of an Enum.
  70. # In this easy example there is an Enum.
  71. class Type(Keyword):
  72. grammar = Enum( K("int"), K("long") )
  73. # Parsing attributes adds them to the resulting thing.
  74. # blank is a callback function. Callback functions are being executed by
  75. # compose(). parse() ignores callback functions. blank inserts " ".
  76. # name() generates a name attribute.
  77. class Parameter(object):
  78. grammar = attr("typing", Type), blank, name()
  79. # We pretty print parameters to remove the class instance name as that can be inconsistent
  80. def __repr__(self):
  81. return f"Symbol({self.name}[{self.typing}]) at 0x{hex(id(self))}"
  82. # A Namespace is a container for named things.
  83. # csl() creates the grammar for a comma separated list.
  84. class Parameters(Namespace):
  85. grammar = optional(csl(Parameter))
  86. # This is an example for a user defined callback function, heading().
  87. # endl is a special callback function. It is never executed. Instead it
  88. # triggers the indention system of compose() and will be replaced by "\n".
  89. class Instruction(str):
  90. def heading(self, parser):
  91. return "/* on level " + str(parser.indention_level) + " */", endl
  92. grammar = heading, word, ";", endl
  93. # indent() is a function which marks things for being indented by compose().
  94. # indent() raises the indention level by 1 for each thing which is inside.
  95. block = "{", endl, maybe_some(indent(Instruction)), "}", endl
  96. # If a thing is a List, then parsed things are being put into.
  97. class Function(List):
  98. grammar = attr("typing", Type), blank, name(), "(", attr("parms", Parameters), ")", endl, block
  99. if __name__ == '__main__':
  100. import doctest
  101. doctest.testmod(optionflags=(doctest.ELLIPSIS | doctest.REPORT_ONLY_FIRST_FAILURE))