用awk把日志内容分段提取

时间：2006-02-22 来源：CUDev

想用awk把下面日志分段提取出来,每段开头都有java.sql.SQL,但不能确保未来所有日志都会以java.sql.SQL开头,可以确定的是每段开头包含字符"Exception",日志样本如下:

java.sql.SQLException: ORA-06502: PL/SQL: 数字或值错误
ORA-06512: 在"CIQUSR.TRT_MIGOODS", line 162
ORA-06512: 在"CIQUSR.TRT_MIDECL_WEB", line 603
ORA-06512: 在line 1

at oracle.jdbc.dbaccess.DBError.throwSqlException(DBError.java:134)
at oracle.jdbc.ttc7.TTIoer.processError(TTIoer.java:289)
at oracle.jdbc.ttc7.Oall7.receive(Oall7.java:573)
at oracle.jdbc.ttc7.TTC7Protocol.doOall7(TTC7Protocol.java:1891)
at oracle.jdbc.ttc7.TTC7Protocol.parseExecuteFetch(TTC7Protocol.java:1093)
at oracle.jdbc.driver.OracleStatement.executeNonQuery(OracleStatement.java:2047)
at oracle.jdbc.driver.OracleStatement.doExecuteOther(OracleStatement.java:1940)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:2709)
at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:589)
at oracle.jdbc.driver.OraclePreparedStatement.execute(OraclePreparedStatement.java:656)
at com.itown.dataexchange.SHTransDataWorker.executePrMid(SHTransDataWorker.java:1088)
at com.itown.dataexchange.SHTransDataWorker.SendDeclData(SHTransDataWorker.java:406)
at com.itown.dataexchange.SHTransDataWorker.doWork(SHTransDataWorker.java:129)
at com.itown.dataexchange.SHTransDataWorker.run(SHTransDataWorker.java:66)
at java.lang.Thread.run(Unknown Source)
java.sql.BatchUpdateException: 批处理中出现错误: ORA-01438: 值大于此列指定的允许精确度

at oracle.jdbc.dbaccess.DBError.throwBatchUpdateException(DBError.java:571)
at oracle.jdbc.dbaccess.DBError.throwBatchUpdateException(DBError.java:626)
at oracle.jdbc.driver.OracleStatement.executeBatch(OracleStatement.java:6841)
at com.itownet.shanghai.sql.PooledStatement.executeBatch(PooledStatement.java:50)
at com.itownet.shanghai.dbaccess.DataHandle.handleTranscation(DataHandle.java:156)
at com.itown.dataexchange.SHTransDataWorker.SendDeclData(SHTransDataWorker.java:400)
at com.itown.dataexchange.SHTransDataWorker.doWork(SHTransDataWorker.java:129)
at com.itown.dataexchange.SHTransDataWorker.run(SHTransDataWorker.java:66)
at java.lang.Thread.run(Unknown Source)
java.sql.SQLException: ORA-06502: PL/SQL: 数字或值错误
ORA-06512: 在"CIQUSR.TRT_MIGOODS", line 162
ORA-06512: 在"CIQUSR.TRT_MIDECL_WEB", line 603
ORA-06512: 在line 1

at oracle.jdbc.dbaccess.DBError.throwSqlException(DBError.java:134)
at oracle.jdbc.ttc7.TTIoer.processError(TTIoer.java:289)
at oracle.jdbc.ttc7.Oall7.receive(Oall7.java:573)
at oracle.jdbc.ttc7.TTC7Protocol.doOall7(TTC7Protocol.java:1891)
at oracle.jdbc.ttc7.TTC7Protocol.parseExecuteFetch(TTC7Protocol.java:1093)
at oracle.jdbc.driver.OracleStatement.executeNonQuery(OracleStatement.java:2047)
at oracle.jdbc.driver.OracleStatement.doExecuteOther(OracleStatement.java:1940)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:2709)
at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:589)
at oracle.jdbc.driver.OraclePreparedStatement.execute(OraclePreparedStatement.java:656)
at com.itown.dataexchange.SHTransDataWorker.executePrMid(SHTransDataWorker.java:1088)
at com.itown.dataexchange.SHTransDataWorker.SendDeclData(SHTransDataWorker.java:406)
at com.itown.dataexchange.SHTransDataWorker.doWork(SHTransDataWorker.java:129)
at com.itown.dataexchange.SHTransDataWorker.run(SHTransDataWorker.java:66)
at java.lang.Thread.run(Unknown Source)
达到的效果是在每段前加上<value>,每段结尾加上</value>,如下
<value>
java.sql.SQLException: ORA-06502: PL/SQL: 数字或值错误
ORA-06512: 在"CIQUSR.TRT_MIGOODS", line 162
ORA-06512: 在"CIQUSR.TRT_MIDECL_WEB", line 603
ORA-06512: 在line 1

      at oracle.jdbc.dbaccess.DBError.throwSqlException(DBError.java:134)
      at oracle.jdbc.ttc7.TTIoer.processError(TTIoer.java:289)
      at oracle.jdbc.ttc7.Oall7.receive(Oall7.java:573)
      at oracle.jdbc.ttc7.TTC7Protocol.doOall7(TTC7Protocol.java:1891)
      at oracle.jdbc.ttc7.TTC7Protocol.parseExecuteFetch(TTC7Protocol.java:1093)
      at oracle.jdbc.driver.OracleStatement.executeNonQuery(OracleStatement.java:2047)
      at oracle.jdbc.driver.OracleStatement.doExecuteOther(OracleStatement.java:1940)
      at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:2709)
      at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:589)
      at oracle.jdbc.driver.OraclePreparedStatement.execute(OraclePreparedStatement.java:656)
      at com.itown.dataexchange.SHTransDataWorker.executePrMid(SHTransDataWorker.java:1088)
      at com.itown.dataexchange.SHTransDataWorker.SendDeclData(SHTransDataWorker.java:406)
      at com.itown.dataexchange.SHTransDataWorker.doWork(SHTransDataWorker.java:129)
      at com.itown.dataexchange.SHTransDataWorker.run(SHTransDataWorker.java:66)
      at java.lang.Thread.run(Unknown Source)
</value>
1. awk实现：

CODE:

# awk
# pickup.awk

BEGIN { count=1 }

$1~/.*Exception:/ { if (count != 1) { print "</value>" } print "<value>";count+=1 }
{ print }

END { print "</value>" } awk -f pickup.awk XXX 解释: BEGIN { count=1 } # 设置变量count

$1~/\w*Exception:/ { if (count != 1) { print "</value>" } print "<value>";count+=1 }

# 如果$1符合正则,判断count是否为1,如果为1说明是第一段记录,那么不用添加</value>结束标志.
# 输出<value>,记数器+1

{ print } # 输出当前行

END { print "</value>" } # 输出结束标志 2. sed实现：

cat filename|sed '/java.sql.*Exception/s/.*/<\/value>\n<value>\n&/' |sed -e '1d' -e '$s/.*/&\n<\/value>/' 解释：这个问题应该就是在含有关键字的行前插入 <value> </value>这2行，解决思路是这样,最后要记得删除第一行的</value>,在最后一行插入</value> 。