Gae Python: Parsing Compressed Xml Exceeds Memory
I am trying to fetch and parse an XML file into a databse. The XML is compressed in GZIP. The GZIP file is ~8MB. When I run the code locally the memory on pythonw.exe builds up to
Solution 1:
What do you expect? First, you read a string into the memory, then - unzip it into the memory, then - construct a DOM tree, still in the memory.
Here are some improvements:
del
every buffer variable the moment you don't need it.- Get rid of DOM XML parser, use event-driven LXML to save memory.
Post a Comment for "Gae Python: Parsing Compressed Xml Exceeds Memory"