easyExcel大数据量导出
com.alibaba.excel.exception.ExcelGenerateException: java.lang.OutOfMemoryError: GC overhead limit exceededat com.alibaba.excel.write.ExcelBuilderImpl.fill(ExcelBuilderImpl.java:84)at com.alibaba.excel.ExcelWriter.fill(ExcelWriter.java:185)
- 原因,easyExcel在进行填充数据的时候比较耗内存容易造成oom
- 解决方案,把数据分批写入文件,部分代码
ExcelWriter excelWriter = null;ByteArrayOutputStream outputStream = new ByteArrayOutputStream();try {FillConfig fillConfig = FillConfig.builder().forceNewRow(true).build();excelWriter = EasyExcel.write(outputStream).withTemplate(templateIn).build();WriteSheet sheet = EasyExcel.writerSheet(0).build();int totalSize = dataList.size();int limit = 20000;int totalPages = totalSize / limit;if (totalPages % limit > 0) {totalPages++;}for (int i = 0; i < totalPages; i++) {if (i == totalPages - 1) {excelWriter.fill(dataList.subList(i * limit, dataList.size()), fillConfig, sheet);} else {excelWriter.fill(dataList.subList(i * limit, limit * (i + 1)), fillConfig, sheet);}}} finally {if (null != excelWriter) {excelWriter.finish();}try {outputStream.close();} catch (IOException e) {log.error("关闭文件流失败");}try {templateIn.close();} catch (IOException e) {log.error("关闭文件流失败");}}
- 验证:分批次写入文件,可以正常导出文件,并未出现oom
- 在导出excel数据之后,fullGc的时候并未第一时间回收数据集合,可以在finally手动回收数据对象
finally {if (!CollectionUtils.isEmpty(dataList)) {dataList.clear();}if (null != excelWriter) {excelWriter.finish();}try {outputStream.close();} catch (IOException e) {log.error("关闭文件流失败");}try {templateIn.close();} catch (IOException e) {log.error("关闭文件流失败");}}
- 在数据行数超过104万条的时候报错

- 1048576是poi的限制
- 超过104万的话可以采用分sheet页导出的方式