log_formatter.py
author Tero Marttila <terom@fixme.fi>
Sun, 13 Sep 2009 00:49:55 +0300
changeset 139 9c7769850195
parent 109 ca82d0fee336
permissions -rw-r--r--
fix error.py version_link_hg argument order
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
     1
"""
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
     2
    Format LogLines into some other representation
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
     3
"""
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
     4
69
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
     5
import re, xml.sax.saxutils
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
     6
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
     7
from log_line import LogTypes
79
43ac75054d5c image formatting \o/
Tero Marttila <terom@fixme.fi>
parents: 73
diff changeset
     8
from log_formatter_pil import PILImageFormatter
80
a0662cff1d9d add RSSFormatter, and link to different types from channel_last.tmpl...
Tero Marttila <terom@fixme.fi>
parents: 79
diff changeset
     9
from log_formatter_rss import RSSFormatter
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
    10
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
    11
class LogFormatter (object) :
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
    12
    """
65
8b50694f841e improve search further
Tero Marttila <terom@fixme.fi>
parents: 64
diff changeset
    13
        Provides a method to format series of LogLines into various output formats, with varying themes.
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
    14
    """
51
07ca28f3a9f2 use improved URLConfig/URLType
Tero Marttila <terom@fixme.fi>
parents: 50
diff changeset
    15
72
5ade0288f2ec implement line-links as UTC timestamps
Tero Marttila <terom@fixme.fi>
parents: 69
diff changeset
    16
    # machine-readable name
51
07ca28f3a9f2 use improved URLConfig/URLType
Tero Marttila <terom@fixme.fi>
parents: 50
diff changeset
    17
    name = None
72
5ade0288f2ec implement line-links as UTC timestamps
Tero Marttila <terom@fixme.fi>
parents: 69
diff changeset
    18
5ade0288f2ec implement line-links as UTC timestamps
Tero Marttila <terom@fixme.fi>
parents: 69
diff changeset
    19
    # human-readable name
5ade0288f2ec implement line-links as UTC timestamps
Tero Marttila <terom@fixme.fi>
parents: 69
diff changeset
    20
    title = None
5ade0288f2ec implement line-links as UTC timestamps
Tero Marttila <terom@fixme.fi>
parents: 69
diff changeset
    21
5ade0288f2ec implement line-links as UTC timestamps
Tero Marttila <terom@fixme.fi>
parents: 69
diff changeset
    22
    ## parameters
5ade0288f2ec implement line-links as UTC timestamps
Tero Marttila <terom@fixme.fi>
parents: 69
diff changeset
    23
    # use a fixed-width font for HTML output
5ade0288f2ec implement line-links as UTC timestamps
Tero Marttila <terom@fixme.fi>
parents: 69
diff changeset
    24
    html_fixedwidth = True
5ade0288f2ec implement line-links as UTC timestamps
Tero Marttila <terom@fixme.fi>
parents: 69
diff changeset
    25
79
43ac75054d5c image formatting \o/
Tero Marttila <terom@fixme.fi>
parents: 73
diff changeset
    26
    def __init__ (self, tz, timestamp_fmt, img_ttf_path, img_font_size) :
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
    27
        """
79
43ac75054d5c image formatting \o/
Tero Marttila <terom@fixme.fi>
parents: 73
diff changeset
    28
            Initialize to format timestamps with the given timezone and timestamp.
43ac75054d5c image formatting \o/
Tero Marttila <terom@fixme.fi>
parents: 73
diff changeset
    29
43ac75054d5c image formatting \o/
Tero Marttila <terom@fixme.fi>
parents: 73
diff changeset
    30
            Use the given TTF font to render image text with the given size, if given, otherwise, a default one.
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
    31
        """
97
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
    32
        
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
    33
        # store
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
    34
        self.tz = tz
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
    35
        self.timestamp_fmt = timestamp_fmt
79
43ac75054d5c image formatting \o/
Tero Marttila <terom@fixme.fi>
parents: 73
diff changeset
    36
        self.img_ttf_path = img_ttf_path
43ac75054d5c image formatting \o/
Tero Marttila <terom@fixme.fi>
parents: 73
diff changeset
    37
        self.img_font_size = img_font_size
97
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
    38
        
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
    39
        # XXX: harcoded
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
    40
        self.date_fmt = '%Y-%m-%d'
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
    41
    
97
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
    42
    def _format_line_text (self, line, template_dict, type=None, full_timestamp=False, **extra) :
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
    43
        """
92
74f6a0b01ddf change debug formatter to to use str(LogLine) for TSV, and fix handling of topic-unset
Tero Marttila <terom@fixme.fi>
parents: 86
diff changeset
    44
            Format the given line as text, using the given { type: string template } dict.
74f6a0b01ddf change debug formatter to to use str(LogLine) for TSV, and fix handling of topic-unset
Tero Marttila <terom@fixme.fi>
parents: 86
diff changeset
    45
            
74f6a0b01ddf change debug formatter to to use str(LogLine) for TSV, and fix handling of topic-unset
Tero Marttila <terom@fixme.fi>
parents: 86
diff changeset
    46
            If type is given, then it overrides line.type
97
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
    47
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
    48
            Any additional keyword args will also be available for the template to use
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
    49
        """
92
74f6a0b01ddf change debug formatter to to use str(LogLine) for TSV, and fix handling of topic-unset
Tero Marttila <terom@fixme.fi>
parents: 86
diff changeset
    50
74f6a0b01ddf change debug formatter to to use str(LogLine) for TSV, and fix handling of topic-unset
Tero Marttila <terom@fixme.fi>
parents: 86
diff changeset
    51
        # default type?
74f6a0b01ddf change debug formatter to to use str(LogLine) for TSV, and fix handling of topic-unset
Tero Marttila <terom@fixme.fi>
parents: 86
diff changeset
    52
        if type is None :
74f6a0b01ddf change debug formatter to to use str(LogLine) for TSV, and fix handling of topic-unset
Tero Marttila <terom@fixme.fi>
parents: 86
diff changeset
    53
            type = line.type
65
8b50694f841e improve search further
Tero Marttila <terom@fixme.fi>
parents: 64
diff changeset
    54
            
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
    55
        # look up the template
97
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
    56
        if type in template_dict :
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
    57
            template = template_dict[type]
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
    58
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
    59
        else :
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
    60
            raise Exception("Format template not defined for type: %s" % LogTypes.name_from_code(type))
65
8b50694f841e improve search further
Tero Marttila <terom@fixme.fi>
parents: 64
diff changeset
    61
        
8b50694f841e improve search further
Tero Marttila <terom@fixme.fi>
parents: 64
diff changeset
    62
        # convert timestamp into display timezone
8b50694f841e improve search further
Tero Marttila <terom@fixme.fi>
parents: 64
diff changeset
    63
        dtz = line.timestamp.astimezone(self.tz)
8b50694f841e improve search further
Tero Marttila <terom@fixme.fi>
parents: 64
diff changeset
    64
        
8b50694f841e improve search further
Tero Marttila <terom@fixme.fi>
parents: 64
diff changeset
    65
        # full timestamps?
8b50694f841e improve search further
Tero Marttila <terom@fixme.fi>
parents: 64
diff changeset
    66
        if full_timestamp :
97
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
    67
            # XXX: let the user define a 'datetime' format instead?
108
d0aca7894fc5 split out helpers._render_type, and add support for ?type= to various other handlers... still needs more work, though
Tero Marttila <terom@fixme.fi>
parents: 97
diff changeset
    68
            timestamp_fmt = self.date_fmt + ' ' + self.timestamp_fmt
65
8b50694f841e improve search further
Tero Marttila <terom@fixme.fi>
parents: 64
diff changeset
    69
8b50694f841e improve search further
Tero Marttila <terom@fixme.fi>
parents: 64
diff changeset
    70
        else :
8b50694f841e improve search further
Tero Marttila <terom@fixme.fi>
parents: 64
diff changeset
    71
            timestamp_fmt = self.timestamp_fmt
86
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
    72
        
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
    73
        # breakdown source
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
    74
        source_nickname, source_username, source_hostname, source_chanflag = line.source
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
    75
        target_nickname = line.target
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
    76
        
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
    77
        # format with dict
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
    78
        return template % dict(
86
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
    79
            channel_name    = line.channel.name,
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
    80
            datetime        = dtz.strftime('%a %b %d %H:%M:%S %Y'),
97
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
    81
            date            = dtz.strftime(self.date_fmt),
86
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
    82
            timestamp       = dtz.strftime(timestamp_fmt),
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
    83
            source_nickname = source_nickname,
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
    84
            source_username = source_username,
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
    85
            source_hostname = source_hostname,
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
    86
            source_chanflag = source_chanflag,
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
    87
            target_nickname = target_nickname,
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
    88
            message         = line.data,
97
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
    89
            **extra
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
    90
        )
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
    91
    
65
8b50694f841e improve search further
Tero Marttila <terom@fixme.fi>
parents: 64
diff changeset
    92
    def format_txt (self, lines, full_timestamps=False) :
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
    93
        """
65
8b50694f841e improve search further
Tero Marttila <terom@fixme.fi>
parents: 64
diff changeset
    94
            Format given lines as plaintext.
8b50694f841e improve search further
Tero Marttila <terom@fixme.fi>
parents: 64
diff changeset
    95
8b50694f841e improve search further
Tero Marttila <terom@fixme.fi>
parents: 64
diff changeset
    96
            If full_timestamps is given, the output will contain full timestamps with both date and time.
8b50694f841e improve search further
Tero Marttila <terom@fixme.fi>
parents: 64
diff changeset
    97
8b50694f841e improve search further
Tero Marttila <terom@fixme.fi>
parents: 64
diff changeset
    98
            No trailing newlines.
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
    99
        """
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
   100
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
   101
        abstract
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
   102
65
8b50694f841e improve search further
Tero Marttila <terom@fixme.fi>
parents: 64
diff changeset
   103
    def format_html (self, lines, full_timestamps=False) :
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
   104
        """
65
8b50694f841e improve search further
Tero Marttila <terom@fixme.fi>
parents: 64
diff changeset
   105
            Format as HTML.
8b50694f841e improve search further
Tero Marttila <terom@fixme.fi>
parents: 64
diff changeset
   106
            
8b50694f841e improve search further
Tero Marttila <terom@fixme.fi>
parents: 64
diff changeset
   107
            See format_txt for information about arguments
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
   108
        """
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
   109
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
   110
        abstract
79
43ac75054d5c image formatting \o/
Tero Marttila <terom@fixme.fi>
parents: 73
diff changeset
   111
    
80
a0662cff1d9d add RSSFormatter, and link to different types from channel_last.tmpl...
Tero Marttila <terom@fixme.fi>
parents: 79
diff changeset
   112
    def format_png (self, lines, full_timestamps=False) :
79
43ac75054d5c image formatting \o/
Tero Marttila <terom@fixme.fi>
parents: 73
diff changeset
   113
        """
43ac75054d5c image formatting \o/
Tero Marttila <terom@fixme.fi>
parents: 73
diff changeset
   114
            Format as a PNG image, returning the binary PNG data
43ac75054d5c image formatting \o/
Tero Marttila <terom@fixme.fi>
parents: 73
diff changeset
   115
        """
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
   116
80
a0662cff1d9d add RSSFormatter, and link to different types from channel_last.tmpl...
Tero Marttila <terom@fixme.fi>
parents: 79
diff changeset
   117
        abstract
a0662cff1d9d add RSSFormatter, and link to different types from channel_last.tmpl...
Tero Marttila <terom@fixme.fi>
parents: 79
diff changeset
   118
    
a0662cff1d9d add RSSFormatter, and link to different types from channel_last.tmpl...
Tero Marttila <terom@fixme.fi>
parents: 79
diff changeset
   119
    def format_rss (self, lines, full_timestamps=False) :
a0662cff1d9d add RSSFormatter, and link to different types from channel_last.tmpl...
Tero Marttila <terom@fixme.fi>
parents: 79
diff changeset
   120
        """
a0662cff1d9d add RSSFormatter, and link to different types from channel_last.tmpl...
Tero Marttila <terom@fixme.fi>
parents: 79
diff changeset
   121
            Format as an XML RSS document
a0662cff1d9d add RSSFormatter, and link to different types from channel_last.tmpl...
Tero Marttila <terom@fixme.fi>
parents: 79
diff changeset
   122
        """
a0662cff1d9d add RSSFormatter, and link to different types from channel_last.tmpl...
Tero Marttila <terom@fixme.fi>
parents: 79
diff changeset
   123
        
a0662cff1d9d add RSSFormatter, and link to different types from channel_last.tmpl...
Tero Marttila <terom@fixme.fi>
parents: 79
diff changeset
   124
        abstract
a0662cff1d9d add RSSFormatter, and link to different types from channel_last.tmpl...
Tero Marttila <terom@fixme.fi>
parents: 79
diff changeset
   125
86
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   126
class BaseHTMLFormatter (LogFormatter) :
69
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   127
    """
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   128
        Implements some HTML-formatting utils
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   129
    """
86
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   130
    
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   131
    # parameters
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   132
    html_fixedwidth = True
69
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   133
86
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   134
    # regexp to match URLs
69
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   135
    URL_REGEXP = re.compile(r"http://\S+")
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   136
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   137
    def _process_links (self, line) :
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   138
        """
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   139
            Processed the rendered line, adding in <a href>'s for things that look like URLs, returning the new line.
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   140
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   141
            The line should already be escaped
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   142
        """
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   143
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   144
        def _encode_url (match) :
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   145
            # encode URL
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   146
            url_html = match.group(0)
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   147
            url_link = xml.sax.saxutils.unescape(url_html)
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   148
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   149
            return '<a href="%(url_link)s">%(url_html)s</a>' % dict(url_link=url_link, url_html=url_html)
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   150
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   151
        return self.URL_REGEXP.sub(_encode_url, line)
86
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   152
 
79
43ac75054d5c image formatting \o/
Tero Marttila <terom@fixme.fi>
parents: 73
diff changeset
   153
    def format_html (self, lines, **kwargs) :
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
   154
        """
72
5ade0288f2ec implement line-links as UTC timestamps
Tero Marttila <terom@fixme.fi>
parents: 69
diff changeset
   155
            Just uses format_txt, but processes links, etc
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
   156
        """
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
   157
        
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
   158
        # format using IrssiTextFormatter
79
43ac75054d5c image formatting \o/
Tero Marttila <terom@fixme.fi>
parents: 73
diff changeset
   159
        for line, txt in self.format_txt(lines, **kwargs) :
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
   160
            # escape HTML
72
5ade0288f2ec implement line-links as UTC timestamps
Tero Marttila <terom@fixme.fi>
parents: 69
diff changeset
   161
            html = xml.sax.saxutils.escape(txt)
69
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   162
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   163
            # process links
72
5ade0288f2ec implement line-links as UTC timestamps
Tero Marttila <terom@fixme.fi>
parents: 69
diff changeset
   164
            html = self._process_links(html)
69
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   165
1f182913b1f2 href-ize urls
Tero Marttila <terom@fixme.fi>
parents: 65
diff changeset
   166
            # yield
72
5ade0288f2ec implement line-links as UTC timestamps
Tero Marttila <terom@fixme.fi>
parents: 69
diff changeset
   167
            yield line, html
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
   168
86
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   169
   
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   170
class IrssiTextFormatter (RSSFormatter, PILImageFormatter, LogFormatter) :
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   171
    """
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   172
        Implements format_txt for irssi-style output
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   173
    """
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   174
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   175
    # format definitions by type
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   176
    __FMT = {
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   177
        LogTypes.RAW        : "%(timestamp)s %(data)s",
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   178
        LogTypes.LOG_OPEN   : "--- Log opened %(datetime)s",
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   179
        LogTypes.LOG_CLOSE  : "--- Log closed %(datetime)s",
97
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
   180
        'DAY_CHANGED'       : "--- Day changed %(date)s",
86
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   181
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   182
        LogTypes.MSG        : "%(timestamp)s <%(source_chanflag)s%(source_nickname)s> %(message)s",
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   183
        LogTypes.NOTICE     : "%(timestamp)s -%(source_nickname)s- %(message)s",
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   184
        LogTypes.ACTION     : "%(timestamp)s  * %(source_nickname)s %(message)s",
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   185
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   186
        LogTypes.JOIN       : "%(timestamp)s -!- %(source_nickname)s [%(source_username)s@%(source_hostname)s] has joined %(channel_name)s",
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   187
        LogTypes.PART       : "%(timestamp)s -!- %(source_nickname)s [%(source_username)s@%(source_hostname)s] has left %(channel_name)s [%(message)s]",
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   188
        LogTypes.KICK       : "%(timestamp)s -!- %(target_nickname)s was kicked from %(channel_name)s by %(source_nickname)s [%(message)s]",
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   189
        LogTypes.MODE       : "%(timestamp)s -!- mode/%(channel_name)s [%(message)s] by %(source_nickname)s",
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   190
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   191
        LogTypes.NICK       : "%(timestamp)s -!- %(source_nickname)s is now known as %(target_nickname)s",
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   192
        LogTypes.QUIT       : "%(timestamp)s -!- %(source_nickname)s [%(source_username)s@%(source_hostname)s] has quit [%(message)s]",
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   193
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   194
        LogTypes.TOPIC      : "%(timestamp)s -!- %(source_nickname)s changed the topic of %(channel_name)s to: %(message)s",
92
74f6a0b01ddf change debug formatter to to use str(LogLine) for TSV, and fix handling of topic-unset
Tero Marttila <terom@fixme.fi>
parents: 86
diff changeset
   195
        'TOPIC_UNSET'       : "%(timestamp)s -!- Topic unset by %(source_nickname)s on %(channel_name)s",
86
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   196
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   197
        LogTypes.SELF_NOTICE: "%(timestamp)s -%(source_nickname)s- %(message)s",
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   198
        LogTypes.SELF_NICK  : "%(timestamp)s -!- %(source_nickname)s is now known as %(target_nickname)s",
97
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
   199
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
   200
        LogTypes.NETSPLIT_START : 
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
   201
                              "%(timestamp)s -!- Netsplit %(source_hostname)s <-> %(target_nickname)s quits: %(_netsplit_targets)s",
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
   202
        LogTypes.NETSPLIT_END :
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
   203
                              "%(timestamp)s -!- Netsplit over, joins: %(_netsplit_targets)s",
86
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   204
    }
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   205
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   206
    def format_txt (self, lines, full_timestamps=False) :
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   207
        # ...handle each line
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   208
        for line in lines :
97
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
   209
            # extra args
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
   210
            extra = {}
109
ca82d0fee336 fix handling of custom types by parser/formatter
Tero Marttila <terom@fixme.fi>
parents: 108
diff changeset
   211
            
ca82d0fee336 fix handling of custom types by parser/formatter
Tero Marttila <terom@fixme.fi>
parents: 108
diff changeset
   212
            # default to line.type
ca82d0fee336 fix handling of custom types by parser/formatter
Tero Marttila <terom@fixme.fi>
parents: 108
diff changeset
   213
            type = line.type
97
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
   214
109
ca82d0fee336 fix handling of custom types by parser/formatter
Tero Marttila <terom@fixme.fi>
parents: 108
diff changeset
   215
            # special formatting for unset-Topic
92
74f6a0b01ddf change debug formatter to to use str(LogLine) for TSV, and fix handling of topic-unset
Tero Marttila <terom@fixme.fi>
parents: 86
diff changeset
   216
            if line.type == LogTypes.TOPIC and line.data is None :
74f6a0b01ddf change debug formatter to to use str(LogLine) for TSV, and fix handling of topic-unset
Tero Marttila <terom@fixme.fi>
parents: 86
diff changeset
   217
                type = 'TOPIC_UNSET'
74f6a0b01ddf change debug formatter to to use str(LogLine) for TSV, and fix handling of topic-unset
Tero Marttila <terom@fixme.fi>
parents: 86
diff changeset
   218
            
97
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
   219
            # format netsplit stuff
109
ca82d0fee336 fix handling of custom types by parser/formatter
Tero Marttila <terom@fixme.fi>
parents: 108
diff changeset
   220
            elif line.type & LogTypes._NETSPLIT_MASK :
97
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
   221
                # format the netsplit-targets stuff
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
   222
                extra['_netsplit_targets'] = line.data
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
   223
86
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   224
            # using __TYPES
97
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
   225
            yield line, self._format_line_text(line, self.__FMT, type, full_timestamps, **extra)
86
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   226
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   227
class IrssiFormatter (BaseHTMLFormatter, IrssiTextFormatter) :
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   228
    """
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   229
        Implements plain black-and-white irssi-style formatting
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   230
    """
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   231
    
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   232
    # name
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   233
    name = 'irssi'
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   234
    title = "Irssi (plain)"
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   235
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   236
class DebugFormatter (BaseHTMLFormatter) :
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   237
    """
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   238
        Implements a raw debug-style formatting of LogLines
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   239
    """
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   240
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   241
    # name
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   242
    name = 'debug'
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   243
    title = "Raw debugging format"
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   244
    
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   245
    def format_txt (self, lines, full_timestamps=False) :
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   246
        # iterate
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   247
        for line in lines :
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   248
            # just dump
97
6165f1ba458d implement parser/formatter netsplits and day-change
Tero Marttila <terom@fixme.fi>
parents: 92
diff changeset
   249
            yield line, unicode(line)
86
645cf9c4441e implement full parser+formatter for irssi
Tero Marttila <terom@fixme.fi>
parents: 80
diff changeset
   250
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
   251
def by_name (name) :
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
   252
    """
64
cdb6403c2498 beginnings of a LogSearchIndex class
Tero Marttila <terom@fixme.fi>
parents: 59
diff changeset
   253
        Lookup and return a class LogFormatter by name
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
   254
    """
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
   255
64
cdb6403c2498 beginnings of a LogSearchIndex class
Tero Marttila <terom@fixme.fi>
parents: 59
diff changeset
   256
    return FORMATTERS[name]
50
f13cf27a360b implement more LogSource features (logs for date, cleanup last_logs), implement irssi parser, formatter, other misc. stuff
Tero Marttila <terom@fixme.fi>
parents:
diff changeset
   257