解析日志文件以获取特定信息

Question

我有一个包含以下信息的日志文件。我需要解析它得到一些信息。我如何使用grep来获取这些信息或其他方法？

connection= 5,size=262144,put=10 get=0 
swift-bench 2013-02-14 16:29:34,913 INFO Auth version: 1.0 
swift-bench 2013-02-14 16:29:36,580 INFO Auth version: 1.0 
swift-bench 2013-02-14 16:29:36,909 INFO 10 PUTS **FINAL** [0 failures], 30.6/s 
swift-bench 2013-02-14 16:29:36,910 INFO Auth version: 1.0 
swift-bench 2013-02-14 16:29:37,028 INFO 10 DEL **FINAL** [0 failures], 86.3/s 

所需的输出：

Connection,size,put,gets,operation,op/s 
5,262144,10,0,PUTS,30.6 
5,262144,10,0,DEL,86.3 

Answer 1

一个使用perl方式：

内容的 script.pl

：

#!/usr/bin/env perl 

use warnings; 
use strict; 

my $nums; 
while (<>) { 
    if ($. == 1) { 
     my @fields = m/(\w+)=/g; 
     push @fields, qw<operation op/s>; 
     printf qq|%s\n|, join q|,|, @fields; 

     $nums = join q|,|, m/=\s*(\d+)/g; 

     next; 
    } 

    my @f = split; 
    if ($f[5] !~ /(?i)version/ and @f > 7) { 
     printf qq|%s\n|, join q|,|, $nums, $f[5], substr($f[ $#f ], 0, length($f[ $#f ]) - 2); 
    } 
} 

而且asumming infile与贴在问题数据，运行它像：

perl script.pl infile 

国债收益率：

connection,size,put,get,operation,op/s 
5,262144,10,0,PUTS,30.6 
5,262144,10,0,DEL,86.3 

Answer 2

#!/bin/bash 
conn=`grep -P -o -e '\d+(?=,size)' logfile` 
size=`grep -P -o -e '(?<=size\=)\d+' logfile` 
put=`grep -P -o -e '(?<=put\=)\d+' logfile` 
get=`grep -P -o -e '(?<=get\=)\d+' logfile` 
for i in `grep -P -e 'INFO \d' logfile | awk '{print $6","$10}' | tr -d '/s'`; do 
echo $conn,$size,$put,$get,$i 
done 

Answer 3

好吧，如果你可以在数据统计所一致的格式如图所示，这将通过玩弄花招与IFS和切碎排队到做位置参数。假设日志文件的名称在命令行上。在parse_swift.pl线17

#!/bin/bash 

logfile=$1 

echo "Connection,size,put,gets,operation,op/s" 
tmpIFS="$IFS" # In case we want to restore IFS later 
IFS="$IFS,=" 
# Note that the read below isn't splitting up the line 
# so the content of IFS isn't a problem 
while read line ; do 
    set -- $line 
    case "$line" in 
     connection*) 
      conn="$2" size="$4" puts="$6" gets="$8" 
     ;; 
     swift-bench*' PUTS '*|swift-bench*' DEL '*) 
      shift 6 
      case "$line" in 
       *'**FINAL**'*) echo "$conn,$size,$puts,$gets,$1,$5" ;; 
       *) echo "$conn,$size,$puts,$gets,$1,$4" ;; 
      esac 
     ;; 
    esac 

done < "$logfile" 

IFS="$tmpIFS" # Not needed if this is the end of the script