Class SimpleLexer

Description

Accepts text and breaks it into tokens.

Some optimisation to make the sure the content is only scanned by the PHP regex parser once. Lexer modes must not start with leading underscores.

Located in /php_parser.php (line 180)


	
			
Direct descendents
Class Description
 class SimpleHtmlLexer Breaks HTML into SAX events.
Method Summary
 SimpleLexer __construct (SimpleSaxParser $parser, [string $start = "accept"], [boolean $case = false])
 void addEntryPattern (string $pattern, string $mode, string $new_mode)
 void addExitPattern (string $pattern, string $mode)
 void addPattern (string $pattern, [string $mode = "accept"])
 void addSpecialPattern (string $pattern, string $mode, string $special)
 string decodeSpecial (string $mode)
 boolean dispatchTokens (string $unmatched, string $matched, [string $mode = false])
 void invokeParser (string $content, boolean $is_match)
 boolean isModeEnd (string $mode)
 boolean isSpecialMode (string $mode)
 void mapHandler (string $mode, string $handler)
 boolean parse (string $raw)
 array/boolean reduce (string $raw)
Methods
Constructor __construct (line 196)

Sets up the lexer in case insensitive matching by default.

  • access: public
SimpleLexer __construct (SimpleSaxParser $parser, [string $start = "accept"], [boolean $case = false])
  • SimpleSaxParser $parser: Handling strategy by reference.
  • string $start: Starting handler.
  • boolean $case: True for case sensitive.

Redefined in descendants as:
addEntryPattern (line 238)

Adds a pattern that will enter a new parsing mode. Useful for entering parenthesis, strings, tags, etc.

  • access: public
void addEntryPattern (string $pattern, string $mode, string $new_mode)
  • string $pattern: Perl style regex, but ( and ) lose the usual meaning.
  • string $mode: Should only apply this pattern when dealing with this type of input.
  • string $new_mode: Change parsing to this new nested mode.
addExitPattern (line 256)

Adds a pattern that will exit the current mode and re-enter the previous one.

  • access: public
void addExitPattern (string $pattern, string $mode)
  • string $pattern: Perl style regex, but ( and ) lose the usual meaning.
  • string $mode: Mode to leave.
addPattern (line 215)

Adds a token search pattern for a particular parsing mode. The pattern does not change the current mode.

  • access: public
void addPattern (string $pattern, [string $mode = "accept"])
  • string $pattern: Perl style regex, but ( and ) lose the usual meaning.
  • string $mode: Should only apply this pattern when dealing with this type of input.
addSpecialPattern (line 278)

Adds a pattern that has a special mode. Acts as an entry and exit pattern in one go, effectively calling a special parser handler for this token only.

  • access: public
void addSpecialPattern (string $pattern, string $mode, string $special)
  • string $pattern: Perl style regex, but ( and ) lose the usual meaning.
  • string $mode: Should only apply this pattern when dealing with this type of input.
  • string $special: Use this mode for this one token.
decodeSpecial (line 399)

Strips the magic underscore marking single token modes.

  • return: Underlying mode name.
  • access: protected
string decodeSpecial (string $mode)
  • string $mode: Mode to decode.
dispatchTokens (line 344)

Sends the matched token and any leading unmatched text to the parser changing the lexer to a new mode if one is listed.

  • return: False if there was any error from the parser.
  • access: protected
boolean dispatchTokens (string $unmatched, string $matched, [string $mode = false])
  • string $unmatched: Unmatched leading portion.
  • string $matched: Actual token match.
  • string $mode: Mode after match. A boolean false mode causes no change.
invokeParser (line 412)

Calls the parser method named after the current mode. Empty content will be ignored. The lexer has a parser handler for each mode in the lexer.

  • access: protected
void invokeParser (string $content, boolean $is_match)
  • string $content: Text parsed.
  • boolean $is_match: Token is recognised rather than unparsed data.
isModeEnd (line 376)

Tests to see if the new mode is actually to leave the current mode and pop an item from the matching mode stack.

  • return: True if this is the exit mode.
  • access: protected
boolean isModeEnd (string $mode)
  • string $mode: Mode to test.
isSpecialMode (line 388)

Test to see if the mode is one where this mode is entered for this token only and automatically leaves immediately afterwoods.

  • return: True if this is the exit mode.
  • access: protected
boolean isSpecialMode (string $mode)
  • string $mode: Mode to test.
mapHandler (line 294)

Adds a mapping from a mode to another handler.

  • access: public
void mapHandler (string $mode, string $handler)
  • string $mode: Mode to be remapped.
  • string $handler: New target handler.
parse (line 308)

Splits the page text into tokens. Will fail

if the handlers report an error or if no content is consumed. If successful then each unparsed and parsed token invokes a call to the held listener.

  • return: True on success, else false.
  • access: public
boolean parse (string $raw)
  • string $raw: Raw HTML text.
reduce (line 434)

Tries to match a chunk of text and if successful removes the recognised chunk and any leading unparsed data. Empty strings will not be matched.

  • return: Three item list of unparsed content followed by the recognised token and finally the action the parser is to take. True if no match, false if there is a parsing error.
  • access: protected
array/boolean reduce (string $raw)
  • string $raw: The subject to parse. This is the content that will be eaten.

Documentation generated on Sun, 31 Oct 2010 16:32:06 -0500 by phpDocumentor 1.4.3