Compare commits

...

28 Commits

Author SHA1 Message Date
Claudio Luck 548dbedf74 make main(), easier merge
3 years ago
Hartmut Goebel 7e7d1b2934 Release 2.6.3
3 years ago
Hartmut Goebel a84b0c9f8a Bump version to 2.6.3.
3 years ago
Hartmut Goebel 661c83c7a8 Update copyright year.
3 years ago
Hartmut Goebel 371febd7de Makefile: Add target `install`.
3 years ago
Hartmut Goebel afef6d8dea Makefile: Add target `dist`.
3 years ago
Hartmut Goebel 9b203022a9 Update .hgignore.
3 years ago
Claudio Luck e783083273 Add a README.
3 years ago
Hartmut Goebel 455a941378 Add setup.py, setup.cfg and MANIFEST.in.
3 years ago
Hartmut Goebel f872862ee9 docs: Include files are always searched at standard location, too.
3 years ago
Hartmut Goebel 4c7ffe5d0d Move data files into the Python package.
3 years ago
Hartmut Goebel 4db2ab6fa7 Catch missing xml2yml.ysl2 early.
3 years ago
Hartmut Goebel af172e42d6 Convert yml2 into a Python package.
3 years ago
Volker Birk f82481c54e date
3 years ago
Volker Birk 22f7358623 Added tag 2.6.2 for changeset f93de5b34430
3 years ago
Volker Birk 01df3fa5e3 explicitely name match attribute
3 years ago
Volker Birk 2bf92f3d1f documentation
3 years ago
Volker Birk ffbf8da61d Added tag 2.6.1 for changeset 6c0e2881db60
3 years ago
Volker Birk 69596b898a bug: textsectionu disappeared
3 years ago
Volker Birk c5c456145c exit on version string
3 years ago
Volker Birk b722d7b93c Added tag 2.6.0 for changeset 9b2eff55a3ae
3 years ago
Volker Birk d3fca48fe5 fixes
3 years ago
Volker Birk 7f4492af25 adapting to Python 3
3 years ago
Volker Birk 4833362d6b Added tag 2.5.10 for changeset c48f0d1d29ac
4 years ago
Volker Birk 8650e9a8aa feature: default function is used as a template
4 years ago
Volker Birk 8213840372 adding hex literals
4 years ago
Volker Birk d9a2d1888f ...
4 years ago
Volker Birk b6b4a93110 adding samples
4 years ago

@ -1,2 +1,7 @@
syntax: glob
*.pyc
*.html
dist/
build/
*.egg-info/
.git/

@ -3,3 +3,8 @@ a0f3d678b3cad40e1ea2db938e6b47ad8fa32a3d 2.5.5
b4343a05f5d584cde0656896f4bf2395c805efcb 2.5.7
172da49a35f62a305bfe06287fc6da74822039b0 2.5.8
8138b86a919635f85bd7c8b195d1035ff85e231b 2.5.9
c48f0d1d29ac0dcf8b912fda56702bbbe5431909 2.5.10
9b2eff55a3aeb3fad6b99ff5b1fd17c2cd6e5f6a 2.6.0
6c0e2881db60c4640d36864e2f0007c8fb2df1ae 2.6.1
f93de5b34430e86b9fd9e9012107f91d6f30d116 2.6.2
b307d3f5b0bff0fb706d3a3f7bd1c03369b7c224 2.6.3

@ -0,0 +1,17 @@
# Files to include in the source package - for the case
# the project directory is are not in a VCS.
include yml2c
include yml2proc
include yml2/*.ysl2
include yml2/*.yml2
include Makefile
include *.yhtml2
include format.css
include gpl-2.0.txt
include samples/*
include vim/syntax/*
exclude .hgtags
exclude .hgignore

@ -1,5 +1,6 @@
YML_PATH=
YML2C=yml2c
PKGVER=$(shell python setup.py -V)
all: homepage
@ -10,11 +11,22 @@ update: homepage format.css gpl-2.0.txt
update-all: update yml2c yml2.py pyPEG.py backend.py yml2proc
if test -z $(VERSION) ; then echo VERSION not set ; exit 1 ; fi
rsync -avC *.py yml2c Makefile yml2proc xml2yml.ysl2 standardlib.ysl2 samples dragon:fdik.org/yml2/
ssh dragon bash -c "cd ; cd fdik.org/; tar cvjf yml-$(VERSION).tar.bz2 yml2/{*.py,*.yml2,*.yhtml2,format.css,gpl-2.0.txt,yml2c,Makefile,yml2proc,xml2yml.ysl2,standardlib.ysl2,samples} ; rm yml2.tar.bz2 ; ln -s yml-$(VERSION).tar.bz2 yml2.tar.bz2"
rsync -avC *.py yml2 yml2c Makefile yml2proc xml2yml.ysl2 standardlib.ysl2 samples dragon:fdik.org/yml2/
ssh dragon bash -c "cd ; cd fdik.org/; tar cvjf yml-$(VERSION).tar.bz2 yml2/{*.py,*.yml2,*.yhtml2,format.css,gpl-2.0.txt,yml2,yml2c,Makefile,yml2proc,xml2yml.ysl2,standardlib.ysl2,samples} ; rm yml2.tar.bz2 ; ln -s yml-$(VERSION).tar.bz2 yml2.tar.bz2"
%.html: %.en.yhtml2 heading.en.yhtml2 homepage.en.yhtml2
$(YML2C) $< -o $@
clean:
rm -f *.html *.pyc *.pyo
rm -f *.html *.pyc *.pyo */*.pyc */*.pyo
rm -rf *.egg-info
dist: dist/yml-$(PKGVER).tar.gz
dist/yml-$(PKGVER).tar.gz:
python setup.py sdist
install:
pip install .
rm setup.py

@ -0,0 +1,144 @@
## What is YML?
Well, it's the idea not to need to define a grammar first when you want to use a Domain Specific Language. For that purpose, YML is being translated into XML. Let's make an example.
Everything which comes close to a C like language, parses without a grammar definition:
This:
template< class T > T max(T a, T b);
Parses to:
<?xml version='1.0' encoding='UTF-8'?>
<template>
<generic>
<class/>
<T/>
</generic>
<T>
<max>
<parm>
<T/>
<a/>
</parm>
<parm>
<T/>
<b/>
</parm>
</max>
</T>
</template>
Instead of defining grammars, you test out and play around until the results are matching your needs. If the resulting tree does not fit what you're expecting, change it by patching the grammar with `decl`:
This:
module A {
interface B {
attribute long n;
};
};
Parses to:
<?xml version='1.0' encoding='UTF-8'?>
<module>
<A>
<interface>
<B>
<attribute>
<long>
<n/>
</long>
</attribute>
</B>
</interface>
</A>
</module>
This does not look like what we want. So we tell YML that we have a module name after the module, an interface name after the interface and type and name after the attribute:
This:
decl module @name;
decl interface @name;
decl attribute @type @name;
module A {
interface B {
attribute long n;
};
};
Parses to:
<?xml version='1.0' encoding='UTF-8'?>
<module name="A">
<interface name="B">
<attribute type="long" name="n"/>
</interface>
</module>
What can I do with YML?
With YML you can:
* use a C-like DSL without writing a grammar first
* generate code out of this DSL using YSLT
* generate code out of UML using YSLT on XMI
* generate code out of any XML based language like SVG using YSLT
* define a wiki like language in just a few lines like YHTML does
* replace bad designed and complicated XML languages with simpler C-like ones
* ... and much more.
How it works: Replacing angle brackets with some Python
Just writing down what I wanted to have instead of XML for a sample:
<list name="List of goods">
<head>
<columTitle>
Goods
</columnTitle>
<columnTitle>
Price
</columnTitle>
</head>
<row>
<value>
Beer
</value>
<value>
20
</value>
</row>
<row>
<value>
Wine
</value>
<value>
30
</value>
</row>
</list>
Something like that should be more easy, say, like this:
list "List of goods" {
head title "Goods", title "Price";
row value "Beer", value 20;
row value "Wine", value 30;
}
### Y Languages
The latter is what I call an Y language a language specified in YML. How could this be achieved? Well, what's to do? To have the required information, how to build XML from the script above, we need:
* the information, that “list of goods” is an attribute named `name`, while `Goods` is the text value of a tag
* `title` shout be written out as `columnTitle`
How to do that? Let's invent a simple definition language for that information:
decl list(name);
decl title alias columnTitle;

@ -975,6 +975,7 @@ page "The Features" {
if the «YML_PATH» environment variable is set to a colon separated list of directories,
these directories are being searched for the given filename. Otherwise, the local
directory is searched.
The system location for «.yml2» and «.ysl2» files is always searched afterwards.
>>
p >>

@ -13,10 +13,12 @@ decl ne is a {
div id=headline {
p > YML Why a Markup Language?!
div class=small {
"YML 2.5.9 of Tu 20 Nov 2018 Copyleft 2007-2018, "
"YML 2.6.3 of Thu 19 Mar 2020 Copyleft 2007-2020, "
a "http://blog.fdik.org", "Volker Birk";
" "
a "http://fdik.org/yml2.tar.bz2", "Download YML 2";
" "
a "https://pep.foundation/dev/repos/yml2/", "Mercurial repository";
}
}

@ -0,0 +1,12 @@
list {
address {
name "Meier";
street "Uhlmannstrasse 42";
village "Laupheim";
}
address {
name "Schmidt";
street "Haupstrasse 23";
village "Bad Waldsee";
}
}

@ -0,0 +1,6 @@
110 DATA "Meier", "Uhlmannstrasse 42", "Laupheim"
120 DATA "Schmidt", "Haupstrasse 23", "Bad Waldsee"
1000 FOR I=1 TO 2 STEP 1
1010 READ NAME$, STREET$, VILLAGE$
1020 PRINT "INSERT INTO address (name, street, village) VALUES ('" + NAME$ + "', '" + STREET$ + "', '" + VILLAGE$ + "');"
1030 NEXT I

@ -0,0 +1,15 @@
include yslt.yml2
tstylesheet {
template "/list" {
apply "address", 0;
| 1000 FOR I=1 TO 2 STEP 1
| 1010 READ NAME$, STREET$, VILLAGE$
| 1020 PRINT "INSERT INTO address (name, street, village) VALUES ('" + NAME$ + "', '" + STREET$ + "', '" + VILLAGE$ + "');"
| 1030 NEXT I
}
template "address" {
| «position() * 10 + 100» DATA "«name»", "«street»", "«village»"
}
}

@ -1,7 +0,0 @@
include idl.yml2
module blub {
interface bla {
int f(in long a, out long b);
}
}

@ -0,0 +1,20 @@
[sdist]
formats=gztar
[metadata]
name = yml2
version = 2.6.3
author = Volker Birk
author_email = vb@pep-project.org
url = https://pep.foundation/dev/repos/yml2/
license = GPL v2
license_file = gpl-2.0.txt
[options]
packages = yml2
scripts =
yml2c
yml2proc
include_package_data = True
zip_safe = False
install_requires = lxml

@ -0,0 +1,6 @@
from setuptools import setup, find_packages
setup(
setup_requires = ["setuptools >= 30.3.0"], # release 2016-12-06
)

@ -87,6 +87,7 @@ page "The YML Toolchain" {
If «YML_PATH» is set to a colon separated ¬http://en.wikipedia.org/wiki/Path_(computing) path¬
list, then the `a href="features#including" code "include";` statement searches these paths.
If «YML_PATH» is not set, the local directory «.» is searched for files.
The system location for «.yml2» and «.ysl2» files is always searched afterwards.
>>
h2 id=processor > The YML/YSLT processor
@ -218,8 +219,6 @@ page "The YML Toolchain" {
p > Convert XML to normalized YML. Equivalent to -xy xml2yml.ysl2
p > The file xml2yml.ysl2 has to be in YML_PATH.
h3 > -V, --version
p >>
@ -233,6 +232,7 @@ page "The YML Toolchain" {
If «YML_PATH» is set to a colon separated ¬http://en.wikipedia.org/wiki/Path_(computing) path¬
list, then the `a href="features#including" code "include";` statement searches these paths.
If «YML_PATH» is not set, the local directory «.» is searched for files.
The system location for «.yml2» and «.ysl2» files is always searched afterwards.
>>
h2 id=recommended > Recommended external tools

@ -1,9 +1,9 @@
# YML 2.5.9 language definition
# YML 2.6.0 language definition
# written by VB.
import re
from pyPEG import keyword, _and, _not
from .pyPEG import keyword, _and, _not
# pyPEG:
#
@ -44,14 +44,14 @@ ppointer = r(r"\*\*" + _symbol, re.U)
macro = r(r"\%" + _symbol, re.U)
reference = r(r"\&" + _symbol, re.U)
NameStartChar = ur''':|[A-Z]|_|[a-z]|[\u00C0-\u00D6]|[\u00D8-\u00F6]|[\u00F8-\u02FF]|[\u0370-\u037D]|[\u037F-\u1FFF]|[\u200C-\u200D]|[\u2070-\u218F]|[\u2C00-\u2FEF]|[\u3001-\uD7FF]|[\uF900-\uFDCF]|[\uFDF0-\uFFFD]'''
NameChar = NameStartChar + ur'''|-|\.|[0-9]|\u00B7|[\u0300-\u036F]|[\u203F-\u2040]'''
NameStartChar = r''':|[A-Z]|_|[a-z]|[\u00C0-\u00D6]|[\u00D8-\u00F6]|[\u00F8-\u02FF]|[\u0370-\u037D]|[\u037F-\u1FFF]|[\u200C-\u200D]|[\u2070-\u218F]|[\u2C00-\u2FEF]|[\u3001-\uD7FF]|[\uF900-\uFDCF]|[\uFDF0-\uFFFD]'''
NameChar = NameStartChar + r'''|-|\.|[0-9]|\u00B7|[\u0300-\u036F]|[\u203F-\u2040]'''
_xmlSymbol = u"(" + NameStartChar + u")(" + NameChar + u")*"
_xmlSymbol = "(" + NameStartChar + ")(" + NameChar + ")*"
xmlSymbol = r(_xmlSymbol)
aliasSymbol = r(ur"-|(" + _xmlSymbol + ur")")
aliasSymbol = r(r"-|(" + _xmlSymbol + r")")
literal = [r(r'""".*?"""', re.S), r(r"'''.*?'''", re.S), r(r"""-?\d+\.\d*|-?\.\d+|-?\d+|".*?"|'.*?'""")]
literal = [r(r'""".*?"""', re.S), r(r"'''.*?'''", re.S), r(r"""0x[a-f0-9]+|-?\d+\.\d*|-?\.\d+|-?\d+|".*?"|'.*?'""")]
filename = [("'", r(r"[^']*"), "'"), ('"', r(r'[^"]*'), '"'), r(r"[^\s;]+")]
ws = r(r"\s+", re.U)

@ -1,20 +1,20 @@
# 2.5.8 backend
# 2.6.1 backend
# written by VB.
import re, codecs
import fileinput
import sys, traceback, exceptions, os
import sys, traceback, os
from xml.sax.saxutils import escape, quoteattr
from copy import deepcopy
from copy import copy, deepcopy
from glob import glob
from pyPEG import code, parse, parseLine, u, Symbol
from yml2 import ymlCStyle, comment, _inner
from .pyPEG import code, parse, parseLine, u, Symbol
from . import ymlCStyle, comment, _inner
ymlFunc, pointers, pythonFunc = {}, {}, {}
in_ns = u""
in_ns = ""
operator = []
included = u""
included = ""
includePath = []
emitlinenumbers = False
encoding = "utf-8"
@ -29,9 +29,9 @@ ymlFunc["operator"] = "#error"
def clearAll():
global ymlFunc, pointers, pythonFunc, in_ns, operator, included
ymlFunc, pointers, pythonFunc = {}, {}, {}
in_ns = u""
in_ns = ""
operator = []
included = u""
included = ""
lq = re.compile(r"\|(\>*)(.*)")
sq = re.compile(r"(\d*)\>(.*)")
@ -48,84 +48,84 @@ def pointer(name):
return u(pointers[name[1:]])
except:
if name == "*_trace_info":
return u'""'
return '""'
if included:
raise LookupError(u"in " + included + u":" + u(line) + u": pointer " + name)
raise LookupError("in " + included + ":" + u(line) + ": pointer " + name)
else:
raise LookupError(u"in " + u(line) + u": pointer " + name)
raise LookupError("in " + u(line) + ": pointer " + name)
def evalPython(expr):
try:
result = eval(u(expr), pythonFunc)
if type(result) is str:
if type(result) is bytes:
return codecs.decode(result, encoding)
else:
return result
except:
name, parm, tb = sys.exc_info()
msg = u"in python expression: " + u(parm)
if name is exceptions.SyntaxError:
msg = "in python expression: " + u(parm)
if name is SyntaxError:
tbl = traceback.format_exception(name, parm, tb)
msg += u"\n" + tbl[-3] + tbl[-2]
msg += "\n" + tbl[-3] + tbl[-2]
else:
msg += u": " + expr + u"\n"
msg += ": " + expr + "\n"
if included:
raise name(u"in " + included + u":" + u(line) + u": " + msg)
raise name("in " + included + ":" + u(line) + ": " + msg)
else:
raise name(u"in " + u(line) + u": " + msg)
raise name("in " + u(line) + ": " + msg)
def execPython(script):
try:
if type(script) is unicode:
exec script in pythonFunc
if type(script) is str:
exec(script, pythonFunc)
else:
exec codecs.decode(script, encoding) in pythonFunc
exec(codecs.decode(script, encoding), pythonFunc)
except:
name, parm, tb = sys.exc_info()
msg = u"in python script: " + u(parm)
if name is exceptions.SyntaxError:
msg = "in python script: " + u(parm)
if name is SyntaxError:
tbl = traceback.format_exception(name, parm, tb)
msg += u"\n" + tbl[-3] + tbl[-2]
msg += "\n" + tbl[-3] + tbl[-2]
else:
msg += u": " + expr + u"\n"
msg += ": " + expr + "\n"
if included:
raise name(u"in " + included + u":" + u(line) + u": " + msg)
raise name("in " + included + ":" + u(line) + ": " + msg)
else:
raise name(u"in " + u(line) + u": " + msg)
raise name("in " + u(line) + ": " + msg)
def textOut(text):
if not text:
return u""
if type(text) is not unicode:
return ""
if type(text) is not str:
text = codecs.decode(text, encoding)
text = text.replace(r'\"', r'\\"')
text = u'u"""' + text.replace('"', r'\"') + u'"""'
text = '"""' + text.replace('"', r'\"') + '"""'
try:
textFunc = ymlFunc["text"]
parms = ['text', ('parm', [text])]
c, result = textFunc(parms)
if c:
if type(textFunc.alias) is unicode:
result += u"</" + textFunc.alias + u">"
if type(textFunc.alias) is str:
result += "</" + textFunc.alias + ">"
else:
result += u"</" + codecs.decode(textFunc.alias, encoding) + u">"
result += "</" + codecs.decode(textFunc.alias, encoding) + ">"
return result
except:
return escape(eval(text))
def strRepl(text):
if not text:
return u""
if type(text) is not unicode:
return ""
if type(text) is not str:
text = codecs.decode(text, encoding)
text = text.replace(r'\"', r'\\"')
text = u'u"""' + text.replace('"', r'\"') + u'"""'
if type(text) is unicode:
text = '"""' + text.replace('"', r'\"') + '"""'
if type(text) is str:
return escape(eval(text))
def applyMacros(macros, text):
result = text
for key, value in macros.iteritems():
for key, value in macros.items():
result = result.replace(key, value)
return result
@ -139,7 +139,7 @@ class YF:
self.pointers = {}
self.macros = {}
if in_ns:
self.alias = in_ns + u":" + name.replace("_", "-")
self.alias = in_ns + ":" + name.replace("_", "-")
else:
self.alias = name.replace("_", "-")
pythonFunc["yml_" + name] = self
@ -205,13 +205,13 @@ class YF:
hasContent = True
if enable_tracing:
text = u(parms) + u", " + u(vals)
pointers["_trace_info"] = u'"' + u(line) + u": " + u(self.name) + u" " + text.replace(u'"', u'#') + u'"'
text = u(parms) + ", " + u(vals)
pointers["_trace_info"] = '"' + u(line) + ": " + u(self.name) + " " + text.replace('"', '#') + '"'
if emitlinenumbers:
global first
if first:
vals["xmlns:yml"] = u"http://fdik.org/yml"
vals["xmlns:yml"] = "http://fdik.org/yml"
first = False
vals["yml:called"] = u(line)
return self.xml(parms, vals, hasContent, avoidTag)
@ -231,7 +231,7 @@ class YF:
self.descends.append(desc)
def addValue(self, parm, value):
if type(value) is str or type(value) is unicode:
if type(value) is str or type(value) is str:
if value[0] != "'" and value[0] != '"':
self.values[parm] = u(value)
else:
@ -241,16 +241,16 @@ class YF:
def xml(self, callParms, callValues, hasContent, avoidTag = False):
global pointers
extraContent = u""
extraContent = ""
if self.content:
hasContent = True
resultParms = self.values.copy()
macros = self.macros.copy()
toDelete = resultParms.keys()
toDelete = [ key for key in resultParms.keys() ]
for key in toDelete:
if key[0] == "*":
del resultParms[key]
for key, value in callValues.iteritems():
for key, value in callValues.items():
if key[0] == "%":
macros[key] = value
else:
@ -261,9 +261,9 @@ class YF:
if self.parms[i][0] == "*":
cp = u(cp)
if "'" in cp:
pointers[self.parms[i][1:]] = u'"' + cp + u'"'
pointers[self.parms[i][1:]] = '"' + cp + '"'
else:
pointers[self.parms[i][1:]] = u"'" + cp + u"'"
pointers[self.parms[i][1:]] = "'" + cp + "'"
elif self.parms[i][0] == "%":
macros[self.parms[i]] = u(cp)
else:
@ -272,21 +272,21 @@ class YF:
extraContent += u(cp)
hasContent = True
i += 1
result = u""
for p, v in resultParms.iteritems():
result = ""
for p, v in resultParms.items():
if p[0] == "'" or p[0] == '"':
p = eval(p)
result += u" "+ p + u"=" + quoteattr(applyMacros(macros, u(v)))
result += " "+ p + "=" + quoteattr(applyMacros(macros, u(v)))
if hasContent:
if avoidTag:
return True, strRepl(extraContent)
else:
return True, u"<" + self.alias + result + u">" + strRepl(extraContent)
return True, "<" + self.alias + result + ">" + strRepl(extraContent)
else:
if avoidTag:
return False, u""
return False, ""
else:
return False, u"<" + self.alias + result + u"/>"
return False, "<" + self.alias + result + "/>"
def replaceContent(tree, subtree):
n = 0
@ -313,7 +313,7 @@ def replaceContent(tree, subtree):
return tree
def executeCmd(text):
if type(text) is not unicode:
if type(text) is not str:
text = codecs.decode(text, encoding)
for (regex, pattern) in operator:
match = re.search(regex, text)
@ -321,11 +321,11 @@ def executeCmd(text):
cmd = pattern
opt = match.groups()
for i in range(len(opt)):
cmd = cmd.replace(u"%" + u(i+1), opt[i])
text = text[:match.start()] + u"`" + cmd + u"`"+ text[match.end():]
cmd = cmd.replace("%" + u(i+1), opt[i])
text = text[:match.start()] + "`" + cmd + "`"+ text[match.end():]
match = re.search(regex, text)
result = u""
result = ""
m = re.search(bq, text)
while text and m:
cmd = m.group(1)
@ -336,9 +336,9 @@ def executeCmd(text):
if rest: raise SyntaxError(cmd)
except SyntaxError:
if included:
raise SyntaxError(u"in " + included + u":" + u(line) + u": syntax error in executing command: " + cmd.strip())
raise SyntaxError("in " + included + ":" + u(line) + ": syntax error in executing command: " + cmd.strip())
else:
raise SyntaxError(u"in " + u(line) + u": syntax error in executing command: " + cmd.strip())
raise SyntaxError("in " + u(line) + ": syntax error in executing command: " + cmd.strip())
inner = _finish(r)
result += head + inner
m = re.search(bq, text)
@ -358,20 +358,20 @@ def codegen(obj):
except: pass
if ctype == "empty":
return code(u"")
return code("")
if ctype == "in_ns":
in_ns = obj[1][0]
subtree = obj[1]
for sel in subtree:
codegen(sel)
in_ns = u""
return code(u"")
in_ns = ""
return code("")
elif ctype == "decl":
name = u""
name = ""
for data in obj[1]:
if type(data) is unicode or type(data) is str:
if type(data) is str:
name = data
try:
yf = ymlFunc[name]
@ -380,7 +380,7 @@ def codegen(obj):
ymlFunc[name] = YF(name)
yf = ymlFunc[name]
if in_ns:
yf.alias = in_ns + u":" + name
yf.alias = in_ns + ":" + name
if not enable_tracing:
if in_ns == "xsl" and (name == "debug" or name=="assert" or name[:7]=="_trace_"):
yf.alias = "-"
@ -394,9 +394,9 @@ def codegen(obj):
yf = ymlFunc[name] = ymlFunc[base].copy(name)
except KeyError:
if included:
raise KeyError(u"in " + included + u":" + u(line) + u": " + base + u" as base for " + name)
raise KeyError("in " + included + ":" + u(line) + ": " + base + " as base for " + name)
else:
raise KeyError(u"in " + u(line) + u": " + base + u" as base for " + name)
raise KeyError("in " + u(line) + ": " + base + " as base for " + name)
elif data[0] == "shape":
shape = ymlFunc[data[1]]
try:
@ -404,9 +404,9 @@ def codegen(obj):
yf.patch(shape)
except KeyError:
if included:
raise KeyError(u"in " + included + u":" + u(line) + u": " + base + u" as shape for " + name)
raise KeyError("in " + included + ":" + u(line) + ": " + base + " as shape for " + name)
else:
raise KeyError(u"in " + u(line) + u": " + base + u" as shape for " + name)
raise KeyError("in " + u(line) + ": " + base + " as shape for " + name)
elif data[0] == "descend":
yf.addDescend(data[1])
elif data[0] == "declParm":
@ -422,23 +422,23 @@ def codegen(obj):
yf.pointers[parmName[1:]] = value
yf.addParm(parmName)
elif parmName[0] == "%":
if type(value) is unicode or type(value) is str:
if type(value) is str:
yf.macros[parmName] = u(evalPython(value))
else:
yf.macros[parmName] = u(evalPython(u(value)))
yf.addParm(parmName)
elif data[0] == "alias":
if in_ns:
yf.alias = in_ns + u":" + data[1][0]
yf.alias = in_ns + ":" + data[1][0]
else:
yf.alias = data[1][0]
elif data[0] == "content":
yf.content = data[1]
return code(u"")
return code("")
elif ctype == "funclist":
result = u""
result = ""
for f in obj[1]:
result += codegen(f)
return code(result)
@ -447,13 +447,13 @@ def codegen(obj):
if len(obj[1]):
return codegen(('func', ['_parentheses', ('content', [obj[1][0]])]))
else:
return u""
return ""
elif ctype == "fparm":
if len(obj[1]):
return codegen(('func', ['_parm', ('content', [obj[1][0]])]))
else:
return u""
return ""
elif ctype == "generic":
return codegen(('func', ['_generic', ('content', [obj[1][0]])]))
@ -468,15 +468,15 @@ def codegen(obj):
if name == "decl":
if ymlFunc[name] == "#error":
if included:
raise SyntaxError(u"in " + included + u":" + u(line) + u": syntax error in decl statement")
raise SyntaxError("in " + included + ":" + u(line) + ": syntax error in decl statement")
else:
raise SyntaxError(u"in " + u(line) + u": syntax error in decl statement")
raise SyntaxError("in " + u(line) + ": syntax error in decl statement")
if name == "define" or name == "operator":
if ymlFunc[name] == "#error":
if included:
raise SyntaxError(u"in " + included + u":" + u(line) + u": syntax error in define statement")
raise SyntaxError("in " + included + ":" + u(line) + ": syntax error in define statement")
else:
raise SyntaxError(u"in " + u(line) + u": syntax error in define statement")
raise SyntaxError("in " + u(line) + ": syntax error in define statement")
if name[0] == "&":
avoidTag = True
@ -485,7 +485,7 @@ def codegen(obj):
if len(name) > 2:
if name[0:2] == "**":
return code(eval('u'+pointer(name[1:])))
return code(eval(''+pointer(name[1:])))
if name[0] == "*":
name = eval(pointer(name))
@ -495,11 +495,15 @@ def codegen(obj):
try:
ymlFunc[name]
except:
except KeyError:
try:
ymlFunc["_"]
return codegen(('func', ['_', ('content', [('funclist', [obj])])]))
except:
if ymlFunc["_"].alias != "-":
return codegen(('func', ['_', ('content', [('funclist', [obj])])]))
else:
ymlFunc[name] = copy(ymlFunc["_"])
ymlFunc[name].alias = name.replace("_", "-")
return codegen(obj)
except KeyError:
ymlFunc[name] = YF(name)
if ymlFunc[name].alias == "-": avoidTag = True
@ -508,9 +512,9 @@ def codegen(obj):
if len(ymlFunc[name].descends):
if obj[1][-1][0] != 'content':
if included:
raise KeyError(u"in " + included + u":" + u(line) + u": " + name + u" has descending attributes, but no descendants are following")
raise KeyError("in " + included + ":" + u(line) + ": " + name + " has descending attributes, but no descendants are following")
else:
raise KeyError(u"in " + u(line) + u": " + name + u" has descending attributes, but no descendants are following")
raise KeyError("in " + u(line) + ": " + name + " has descending attributes, but no descendants are following")
def first_func(obj):
if type(obj) is tuple or type(obj) is Symbol:
@ -559,7 +563,7 @@ def codegen(obj):
if dname[0] == "*":
pointers[dname[1:]] = "'" + f[1][0] + "'"
else:
add_params.append( ('parm', [dname, u"'" + f[1][0] + u"'"]) )
add_params.append( ('parm', [dname, "'" + f[1][0] + "'"]) )
try:
add_params.extend( get_parms(f) )
except: pass
@ -571,14 +575,14 @@ def codegen(obj):
to_add.append( ('func', new_things ) )
except:
if included:
raise KeyError(u"in " + included + u":" + u(line) + u": " + name + u" has descending attributes, and too less descendants are following")
raise KeyError("in " + included + ":" + u(line) + ": " + name + " has descending attributes, and too less descendants are following")
else:
raise KeyError(u"in " + u(line) + u": " + name + u" has descending attributes, and too less descendants are following")
raise KeyError("in " + u(line) + ": " + name + " has descending attributes, and too less descendants are following")
if not to_add:
to_add = ( obj, )
complete = u""
complete = ""
for obj in to_add:
subtree = None
@ -602,14 +606,14 @@ def codegen(obj):
result += codegen(sel)
if hasContent and not(avoidTag):
result += u"</" + ymlFunc[name].alias + u">"
result += "</" + ymlFunc[name].alias + ">"
complete += result
return code(complete)
elif ctype == "textsection":
result = u''
result = ''
ll = obj[1].splitlines()
space = len(ll[-1]) - 2
for l in ll[1:-1]:
@ -622,15 +626,15 @@ def codegen(obj):
result += _finish(r)
except SyntaxError:
if included:
raise SyntaxError(u"in " + included + u":" + u(line) + u": syntax error in executing command: " + cmd.strip())
raise SyntaxError("in " + included + ":" + u(line) + ": syntax error in executing command: " + cmd.strip())
else:
raise SyntaxError(u"in " + u(line) + u": syntax error in executing command: " + cmd.strip())
raise SyntaxError("in " + u(line) + ": syntax error in executing command: " + cmd.strip())
else:
result += codegen(Symbol(u'lineQuote', u'| ' + l[space:]))
result += codegen(Symbol('lineQuote', '| ' + l[space:]))
return code(result)
elif ctype == "textsectionu":
result = u''
result = ''
ll = obj[1].splitlines()
space = len(ll[-1]) - 2
for l in ll[1:-1]:
@ -643,16 +647,16 @@ def codegen(obj):
result += _finish(r)
except SyntaxError:
if included:
raise SyntaxError(u"in " + included + u":" + u(line) + u": syntax error in executing command: " + cmd.strip())
raise SyntaxError("in " + included + ":" + u(line) + ": syntax error in executing command: " + cmd.strip())
else:
raise SyntaxError(u"in " + u(line) + u": syntax error in executing command: " + cmd.strip())
raise SyntaxError("in " + u(line) + ": syntax error in executing command: " + cmd.strip())
else:
if result != u'': result += u' '
result += codegen(Symbol(u'quote', [u'> ' + l[space:]]))
if result != '': result += ' '
result += codegen(Symbol('quote', ['> ' + l[space:]]))
return code(result)
elif ctype == "lineQuote" or ctype == "quote":
m, text, base, inds = None, u"", 0, 0
m, text, base, inds = None, "", 0, 0
if ctype == "lineQuote":
text = obj[1]
@ -670,20 +674,20 @@ def codegen(obj):
inds = int(m.group(1))
text = m.group(2)[1:]
else:
if type(text) is unicode or type(text) is str:
if type(text) is str:
text = u(evalPython(text))
ind = u""
ind = ""
if inds > -1:
try:
cmd = evalPython(u"indent(" + u(inds) + u")")
cmd = evalPython("indent(" + u(inds) + ")")
result, rest = parseLine(u(cmd), _inner, [], True, comment)
if rest:
raise SyntaxError()
ind = _finish(result)
except: pass
if ctype == "lineQuote": text += u"\n"
if ctype == "lineQuote": text += "\n"
hasTextFunc = False
try:
@ -697,13 +701,13 @@ def codegen(obj):
elif ctype == "tagQuote":
m = tq.match(obj[1])
if m.group(1) == "<":
return code(u"<" + m.group(2))
return code("<" + m.group(2))
else:
return code(m.group(2))
elif ctype == "operator":
operator.append((re.compile(evalPython(obj[1][0])), obj[1][1]))
return code(u"")
return code("")
elif ctype == "constant":
name = obj[1][0]
@ -711,7 +715,7 @@ def codegen(obj):
name = name[1:]
value = obj[1][1]
pointers[name] = value
return code(u"")
return code("")
elif ctype == "include":
reverse = False
@ -724,7 +728,7 @@ def codegen(obj):
ktext = True
elif arg[0] == "kxml":
kxml = True
elif type(arg) is unicode or type(arg) is str:
elif type(arg) is str:
filemask = arg
if filemask[0] == '/' or filemask[0] == '.':
@ -740,14 +744,14 @@ def codegen(obj):
if not(files):
if included:
raise IOError(u"in " + included + ":" + u(line) + u": include file(s) '" + filemask + u"' not found")
raise IOError("in " + included + ":" + u(line) + ": include file(s) '" + filemask + "' not found")
else:
raise IOError(u"in " + u(line) + u": include file(s) '" + filemask + u"' not found")
raise IOError("in " + u(line) + ": include file(s) '" + filemask + "' not found")
includeFile = fileinput.input(files, mode="rU", openhook=fileinput.hook_encoded(encoding))
_included = included
if ktext or kxml:
text = u""
text = ""
for line in includeFile:
included = includeFile.filename()
if kxml:
@ -776,7 +780,7 @@ def codegen(obj):
parms = []
data = obj[1]
for p in data:
if type(p) is unicode or type(p) is str:
if type(p) is str:
name = p
elif type(p) is tuple or type(p) is Symbol:
ptype = p[0]
@ -786,9 +790,9 @@ def codegen(obj):
else:
parms.append(p[1][0])
if len(parms) == 0:
exp = name + u"()"
exp = name + "()"
elif len(parms) == 1:
exp = name + u"(" + u(parms[0]) + u")"
exp = name + "(" + u(parms[0]) + ")"
else:
exp = name + u(tuple(parms))
cmd = evalPython(exp)
@ -798,31 +802,31 @@ def codegen(obj):
return code(_finish(result))
else:
return code(u"")
return code("")
def _finish(tree):
result = u""
python = u""
result = ""
python = ""
for el in tree:
if el[0] == "python":
if el[1][0][:2] == "!!":
python += el[1][0][2:-2]
else:
python += el[1][0][1:] + u"\n"
python += el[1][0][1:] + "\n"
continue
else:
if python:
execPython(python)
python = u""
python = ""
try:
result += codegen(el)
except RuntimeError:
if included:
raise RuntimeError(u"in " + included + u":" + u(line))
raise RuntimeError("in " + included + ":" + u(line))
else:
raise RuntimeError(u"in " + u(line))
raise RuntimeError("in " + u(line))
if python:
execPython(python)

@ -1,13 +1,13 @@
# YPL parser 1.5
# YPL parser 1.6
# adapted for Python 3.x
# written by VB.
import re
import sys, codecs
import exceptions
class keyword(unicode): pass
class code(unicode): pass
class keyword(str): pass
class code(str): pass
class ignore(object):
def __init__(self, regex_text, *args):
self.regex = re.compile(regex_text, *args)
@ -18,10 +18,10 @@ class _and(object):
class _not(_and): pass
class Name(unicode):
class Name(str):
def __init__(self, *args):
self.line = 0
self.file = u""
self.file = ""
class Symbol(list):
def __init__(self, name, what):
@ -31,27 +31,27 @@ class Symbol(list):
self.append(what)
def __call__(self):
return self.what
def __unicode__(self):
return u'Symbol(' + repr(self.__name__) + ', ' + repr(self.what) + u')'
def __str__(self):
return 'Symbol(' + repr(self.__name__) + ', ' + repr(self.what) + ')'
def __repr__(self):
return unicode(self)
return str(self)
word_regex = re.compile(ur"\w+")
rest_regex = re.compile(ur".*")
word_regex = re.compile(r"\w+")
rest_regex = re.compile(r".*")
print_trace = False
def u(text):
if isinstance(text, exceptions.BaseException):
if isinstance(text, BaseException):
text = text.args[0]
if type(text) is unicode:
if type(text) is str:
return text
if isinstance(text, str):
if isinstance(text, bytes):
if sys.stdin.encoding:
return codecs.decode(text, sys.stdin.encoding)
else:
return codecs.decode(text, "utf-8")
return unicode(text)
return str(text)
def skip(skipper, text, skipWS, skipComments):
if skipWS:
@ -104,7 +104,7 @@ class parser(object):
if print_trace:
try:
if _pattern.__name__ != "comment":
sys.stderr.write(u"match: " + _pattern.__name__ + u"\n")
sys.stderr.write("match: " + _pattern.__name__ + "\n")
except: pass
if self.restlen == -1:
@ -146,7 +146,7 @@ class parser(object):
if print_trace:
try:
if pattern.__name__ != "comment":
sys.stderr.write(u"testing with " + pattern.__name__ + u": " + textline[:40] + u"\n")
sys.stderr.write("testing with " + pattern.__name__ + ": " + textline[:40] + "\n")
except: pass
if pattern.__name__[0] != "_":
@ -160,7 +160,7 @@ class parser(object):
pattern_type = type(pattern)
if pattern_type is str or pattern_type is unicode:
if pattern_type is str:
if text[:len(pattern)] == pattern:
text = skip(self.skipper, text[len(pattern):], skipWS, skipComments)
return R(None, text)
@ -251,11 +251,11 @@ class parser(object):
syntaxError()
else:
raise SyntaxError(u"illegal type in grammar: " + u(pattern_type))
raise SyntaxError("illegal type in grammar: " + u(pattern_type))
def lineNo(self):
if not(self.lines): return u""
if self.restlen == -1: return u""
if not(self.lines): return ""
if self.restlen == -1: return ""
parsed = self.textlen - self.restlen
left, right = 0, len(self.lines)
@ -266,20 +266,20 @@ class parser(object):
try:
if self.lines[mid + 1][0] >= parsed: