1

我想读取大的 xls 或 xlsx 文件（大约超过 30 MB 并且有 70,000+ 行）。我可以很容易地使用 Apache POI 读取小的 excel 文件，直到出现 OutOfMemory 错误。

性能和内存使用是我关心的问题。我阅读了许多帖子，如果内存占用是一个问题，那么对于 XSSF，您可以获取底层 XML 数据，并使用 XSSF 和 SAX（事件 API）自己处理它。好吧，我发现它很有趣，现在可以毫无问题地读取整个 xlsx 文件。在不使用事件 API 时，它消耗的内存（小于 70 MB）与几乎以 GB 为单位相比（如果我将 -Xmx 设置为 1024m 并且它仍然过去挂起，则上升到 1GB）要少得多。

但现在我想自定义读取过程，只允许从 excel 中读取特定的行。我可以使用 org.apache.poi.ss.usermodel.Sheet#getRow(int rownum) 轻松做到这一点。但是使用事件 API 它读取所有行没有任何中断，我发现很难读取特定的行，例如只有行号 2、3、5 等。下面是我的整个代码：

import java.io.InputStream;
import java.util.Iterator;
import java.util.Vector;
import org.apache.poi.xssf.eventusermodel.XSSFReader;
import org.apache.poi.xssf.model.SharedStringsTable;
import org.apache.poi.xssf.usermodel.XSSFRichTextString;
import org.apache.poi.openxml4j.opc.OPCPackage;
import org.xml.sax.Attributes;
import org.xml.sax.ContentHandler;
import org.xml.sax.InputSource;
import org.xml.sax.SAXException;
import org.xml.sax.XMLReader;
import org.xml.sax.helpers.DefaultHandler;
import org.xml.sax.helpers.XMLReaderFactory;

/**
 * XSSF and SAX (Event API)
 */
public class FromHowTo {
    public void processAllSheets(String filename) throws Exception {
        OPCPackage pkg = OPCPackage.open(filename);
        XSSFReader r = new XSSFReader( pkg );
        SharedStringsTable sst = r.getSharedStringsTable();

        XMLReader parser = fetchSheetParser(sst);

        Iterator<InputStream> sheets = r.getSheetsData();
        while(sheets.hasNext()) {
            InputStream sheet = sheets.next();
            InputSource sheetSource = new InputSource(sheet);
            parser.parse(sheetSource);
            sheet.close();
        }
    }

    public XMLReader fetchSheetParser(SharedStringsTable sst) throws SAXException {
        XMLReader parser = XMLReaderFactory.createXMLReader("org.apache.xerces.parsers.SAXParser");
        ContentHandler handler = new SheetHandler(sst);
        parser.setContentHandler(handler);
        return parser;
    }

    /** 
     * See org.xml.sax.helpers.DefaultHandler javadocs 
     */
    private static class SheetHandler extends DefaultHandler {
        private SharedStringsTable sst;
        private String lastContents;
        private boolean nextIsString;
        Vector values = new Vector(10);

        private SheetHandler(SharedStringsTable sst) {
            this.sst = sst;
        }

        public void startElement(String uri, String localName, String name, Attributes attributes) throws SAXException {
            // c => cell

            if(name.equals("c")) {
                // Figure out if the value is an index in the SST
                String cellType = attributes.getValue("t");
                //System.out.println(cellType);
                if(cellType != null && cellType.equals("s")) {
                    nextIsString = true;
                } else {
                    nextIsString = false;
                }
            }
            // Clear contents cache
            lastContents = "";
        }

        public void endElement(String uri, String localName, String name) throws SAXException {
            // Process the last contents as required.
            // Do now, as characters() may be called more than once
            if(nextIsString) {
                try {
                    int idx = Integer.parseInt(lastContents);
                    lastContents = new XSSFRichTextString(sst.getEntryAt(idx)).toString();
                } catch (NumberFormatException e) {
                }
            }

            // v => contents of a cell
            // Output after we've seen the string contents
            if(name.equals("v")) {
                values.add(lastContents);
            }

            if(name.equals("row")) {
                System.out.println(values);
                values.removeAllElements();
            }
        }

        public void characters(char[] ch, int start, int length) throws SAXException {
            lastContents += new String(ch, start, length);
        }
    }

    public static void main(String[] args) throws Exception {
        FromHowTo howto = new FromHowTo();
        howto.processAllSheets(args[0]);
    }
}

我将 JRE7 与 Apache POI 3.7 一起使用。有人可以帮助我使用 Event API 获取特定行吗？

4

1 回答 1

5

each row start element has a row number. it can be retrieved from the attributes

long rowIndex = Long.valueOf(attributes.getValue("r"));

The event model will go through to all rows but you can get he index and handle your data accordingly in the endElement

于 2013-03-04T18:02:20.787 回答