EXPECT(1)

NAME
Expect.pm - Expect for Perl

VERSION
1.12

SYNOPSIS
  use Expect;
  # create an Expect object by spawning another process
  my $exp = Expect->spawn($command, @params);
  # or by using an already opened filehandle
  my $exp = Expect->exp_init(\*FILEHANDLE);
  # if you prefer the OO mindset:
  my $exp = new Expect ($command, @parameters);
  # send some string there:
  $exp->send("string\r");
  # or, for the filehandle mindset:
  print $exp "string\r";
  # then do some pattern matching with either the simple interface
  $exp->expect($timeout, @match_patterns);
  # or multi-match on several spawned commands at once
  expect($timeout,
         '-i' => $exp1, @match_patterns1,
         '-i' => \@exp_list, @match_patterns234,
        );
  # even with callbacks, just like the Tcl version
  $exp->expect($timeout,
               [ qr/regex1/ => sub { my $exp = shift;
                                     $exp->send("response\n");
                                     exp_continue; } ],
               [ "regexp2" , \&callback, @cbparms ],
              );
  # if no longer needed, do a soft_close to nicely shut down the command
  $exp->soft_close();
  # or be less patient with
  $exp->hard_close();
Expect.pm is built to either spawn a process or take an existing filehandle and interact with it such that normally interactive tasks can be done without operator assistance. This concept makes more sense if you are already familiar with the versatile Tcl version of Expect. The public functions that make up Expect.pm are:

  Expect::interconnect(@objects_to_be_read_from)
  Expect::test_handles($timeout, @objects_to_test)
  Expect::version($version_requested | undef);
  $object->clear_accum()
  $object->debug($debug_level)
  $object->exp_internal(0 | 1 | undef)
  $object->exp_stty($stty_modes) # See the IO::Stty docs
  $object->before();
  $object->match();
  $object->after();
  $object->matchlist();
  $object->match_number();
  $object->error();
  $object->command();
  $object->exitstatus();
  $object->pty_handle();
  $object->do_soft_close();
  $object->restart_timeout_upon_receive(0 | 1);
  $object->interact($other_object, $escape_sequence)
  $object->log_group(0 | 1 | undef)
  $object->log_user(0 | 1 | undef)
  $object->log_file("filename" | $filehandle | \&coderef | undef)
  $object->manual_stty(0 | 1 | undef)
  $object->match_max($max_buffersize or undef)
  $object->pid();
  $object->send_slow($delay, @strings_to_send)
  $object->set_group(@listen_group_objects | undef)
  $object->set_seq($sequence,\&function,\@parameters);
There are several configurable package variables that affect the behavior of Expect. They are:

  $Expect::Debug;
  $Expect::Exp_Internal;
  $Expect::Log_Group;
  $Expect::Log_Stdout;
  $Expect::Manual_Stty;
  $Expect::Multiline_Matching;
  $Expect::Do_Soft_Close;

DESCRIPTION
The Expect module is a successor of Comm.pl and a descendent of Chat.pl. It more closely ressembles the Tcl Expect language than its predecessors. It does not contain any of the networking code found in Comm.pl. I suspect this would be obsolete anyway given the advent of IO::Socket and external tools such as netcat.

Expect.pm is an attempt to have more of a switch() & case feeling to make decision processing more fluid. Three separate types of debugging have been implemented to make code production easier.

It is now possible to interconnect multiple file handles (and processes) much like Tcl's expect. An attempt was made to enable all the features of Tcl's expect without forcing Tcl on the victim programmer :-) .

USAGE
Initializes $new_handle_object for use with other Expect functions. It must be passed a _reference_ to FILEHANDLE if you want it to work properly. IO::File objects are preferable. Returns a reference to the newly created object. Forks and execs $command. Returns a reference to the newly created process object. Returns undef if the fork was unsuccessful or the object could not be created.

Note that spawn() passes its parameters unchanged to Perls exec(), so look there for semantics. Sets debug level for $object. 1 refers to general debugging information, 2 refers to verbose debugging and 0 refers to no debugging. If you call debug() with no parameters it will return the current debugging level. When the object is created the debugging level will match that $Expect::Debug, normally 0.

The '3' setting is new with 1.05, and adds the additional functionality of having the _full_ accumulated buffer printed every time data is read from an Expect object. This was implemented by request. I recommend against using this unless you think you need it as it can create quite a quantity of output under some circumstances.. Sets/unsets 'exp_internal' debugging. This is similar in nature to its Tcl counterpart. It is extremely valuable when debugging expect() sequences. When the object is created the exp_internal setting will match the value of $Expect::Exp_Internal, normally 0. Returns the current setting if called without parameters. It is highly recommended that you make use of the debugging features lest you have angry code. Sets the tty mode for $object's associated terminal to $mode. Typical values are 'sane', 'raw', and 'raw -echo'. This should be used in conjunction with manual_stty(). See the docs for IO::Stty to see what values make sense here. or, more like Tcl/Expect,

  expect($timeout,
         '-i', [ $obj1, $obj2, ... ],
               [ $re_pattern, sub { ...; exp_continue; }, @subparms, ],
               [ 'eof', sub { ... } ],
               [ 'timeout', sub { ... }, \$subparm1 ],
         '-i', [ $objn, ...],
               '-ex', $exact_pattern, sub { ... },
               $exact_pattern, sub { ...; exp_continue_timeout; },
               '-re', $re_pattern, sub { ... },
         '-i', \@object_list, @pattern_list,
         ...);
Simple interface:

Given $timeout in seconds Expect will wait for $object's handle to produce one of the match_patterns. Due to o/s limitations $timeout should be a round number. If $timeout is 0 Expect will check one time to see if $object's handle contains any of the match_patterns. If $timeout is undef Expect will wait forever for a pattern to match.

If called in a scalar context, expect() will return the position of the matched pattern within $match_patterns, or undef if no pattern was matched. This is a position starting from 1, so if you want to know which of an array of @matched_patterns matched you should subtract one from the return value.

If called in an array context expect() will return ($matched_pattern_position, $error, $successfully_matching_string, $before_match, and $after_match).

$matched_pattern_position will contain the value that would have been returned if expect() had been called in a scalar context. $error is the error that occurred that caused expect() to return. $error will contain a number followed by a string equivalent expressing the nature of the error. Possible values are undef, indicating no error, '1:TIMEOUT' indicating that $timeout seconds had elapsed without a match, '2:EOF' indicating an eof was read from $object, '3: spawn id($fileno) died' indicating that the process exited before matching and '4:$!' indicating whatever error was set in $ERRNO during the last read on $object's handle. All handles indicated by set_group plus STDOUT will have all data to come out of $object printed to them during expect() if log_group and log_stdout are set.

Changed from older versions is the regular expression handling. By default now all strings passed to expect() are treated as literals. To match a regular expression pass '-re' as a parameter in front of the pattern you want to match as a regexp.

Example:

  $object->expect(15, 'match me exactly','-re','match\s+me\s+exactly');
This change makes it possible to match literals and regular expressions in the same expect() call.

Also new is multiline matching. ^ will now match the beginning of lines. Unfortunately, because perl doesn't use $/ in determining where lines break using $ to find the end of a line frequently doesn't work. This is because your terminal is returning "\r\n" at the end of every line. One way to check for a pattern at the end of a line would be to use \r?$ instead of $.

Example: Spawning telnet to a host, you might look for the escape character. telnet would return to you "\r\nEscape character is '^]'.\r\n". To find this you might use $match='^Escape char.*\.\r?$';

  $telnet->expect(10,'-re',$match);
New more TCL-like interface (added by RGiersig@cpan.org):

It's now possible to expect on more than one connection at a time by specifying '`-i'' (see the TCL expect manpage) and a ref to an array containing Expect objects, e.g.

 expect($timeout,
        '-i', $exp1, @patterns_1,
        '-i', [ $exp2, $exp3 ], @patterns_2_3,
       )
Furthermore, patterns can now be specified as array refs containing [$regexp, sub { ...}, @optional_subprams] . When the pattern matches, the subroutine is called with parameters ($matched_expect_obj, @optional_subparms). The subroutine can return the symbol `exp_continue' to continue the expect matching with timeout starting anew or return the symbol `exp_continue_timeout' for continuing expect without resetting the timeout count.

 $exp->expect($timeout,
              [ qr/username: /i, sub { my $self = shift;
                                       $self->send("$username\r");
                                       exp_continue; }],
              [ qr/password: /i, sub { my $self = shift;
                                       $self->send("$password\r");
                                       exp_continue; }],
              $shell_prompt);
`expect' is now exported by default. before() returns the 'before' part of the last expect() call. If the last expect() call didn't match anything, exp_before() will return the entire output of the object accumulated before the expect() call finished. returns the 'after' part of the last expect() call. If the last expect() call didn't match anything, exp_after() will return undef(). returns the string matched by the last expect() call, undef if no string was matched. exp_match_number() returns the number of the pattern matched by the last expect() call. Keep in mind that the first pattern in a list of patterns is 1, not 0. Returns undef if no pattern was matched. exp_matchlist() returns a list of matched substrings from the brackets () inside the regexp that last matched. ($object->matchlist)[0] thus corresponds to $1, ($object->matchlist)[1] to $2, etc. exp_error() returns the error generated by the last expect() call if no pattern was matched. It is typically useful to examine the value returned by before() to find out what the output of the object was in determining why it didn't match any of the patterns. Clear the contents of the accumulator for $object. This gets rid of any residual contents of a handle after expect() or send_slow() such that the next expect() call will only see new data from $object. The contents of the accumulator are returned. exp_command() returns the string that was used to spawn the command. Helpful for debugging and for reused patternmatch subroutines. Returns the exit status of $object (if it already exited). Returns a string representation of the attached pty, for example: `spawn id(5)' (pty has fileno 5), `handle id(7)' (pty was initialized from fileno 7) or `STDIN'. Useful for debugging. If this is set to 1, the expect timeout is retriggered whenever something is received from the spawned command. This allows to perform some aliveness testing and still expect for patterns.

    $exp->restart_timeout_upon_receive(1);
    $exp->expect($timeout,
                 [ timeout => \&report_timeout ],
                 [ qr/pattern/ => \&handle_pattern],
                );
Now the timeout isn't triggered if the command produces any kind of output, i.e. is still alive, but you can act upon patterns in the output. Read from @objects and print to their @listen_groups until an escape sequence is matched from one of @objects and the associated function returns 0 or undef. The special escape sequence 'EOF' is matched when an object's handle returns an end of file. Note that it is not necessary to include objects that only accept data in @objects since the escape sequence is _read_ from an object. Further note that the listen_group for a write-only object is always empty. Why would you want to have objects listening to STDOUT (for example)? By default every member of @objects _as well as every member of its listen group_ will be set to 'raw -echo' for the duration of interconnection. Setting $object->manual_stty() will stop this behavior per object. The original tty settings will be restored as interconnect exits. Given a set of objects determines which objects' handles have data ready to be read. Returns an array who's members are positions in @objects that have ready handles. Returns undef if there are no such handles ready. Returns current version of Expect. As of .99 earlier versions are not supported. Too many things were changed to make versioning possible. interact() is essentially a macro for calling interconnect() for connecting 2 processes together. \*FILEHANDLE defaults to \*STDIN and $escape_sequence defaults to undef. Interaction ceases when $escape_sequence is read from FILEHANDLE, not $object. $object's listen group will consist solely of \*FILEHANDLE for the duration of the interaction. \*FILEHANDLE will not be echoed on STDOUT. Set/unset logging of $object to its 'listen group'. If set all objects in the listen group will have output from $object printed to them during $object->expect(), $object->send_slow(), and `Expect::interconnect($object , ...)'. Default value is on. During creation of $object the setting will match the value of $Expect::Log_Group, normally 1. Set/unset logging of object's handle to STDOUT. This corresponds to Tcl's log_user variable. Returns current setting if called without parameters. Default setting is off for initialized handles. When a process object is created (not a filehandle initialized with exp_init) the log_stdout setting will match the value of $Expect::Log_Stdout variable, normally 1. If/when you initialize STDIN it is usually associated with a tty which will by default echo to STDOUT anyway, so be careful or you will have multiple echoes. Log session to a file. All characters send to or received from the spawned process are written to the file. Normally appends to the logfile, but you can pass an additional mode of "w" to truncate the file upon open():

  $object->log_file("filename", "w");
Returns the logfilehandle.

If called with an undef value, stops logging and closes logfile:

  $object->log_file(undef);
If called without argument, returns the logfilehandle:

  $fh = $object->log_file();
Can be set to a code ref, which will be called instead of printing to the logfile:

  $object->log_file(\&myloggerfunc);
During Expect->interconnect() if $sequence is read from $object &function will be executed with parameters @function_parameters. It is _highly recommended_ that the escape sequence be a single character since the likelihood is great that the sequence will be broken into to separate reads from the $object's handle, making it impossible to strip $sequence from getting printed to $object's listen group. \&function should be something like 'main::control_w_function' and @function_parameters should be an array defined by the caller, passed by reference to set_seq(). Your function should return a non-zero value if execution of interconnect() is to resume after the function returns, zero or undefined if interconnect() should return after your function returns. The special sequence 'EOF' matches the end of file being reached by $object. See interconnect() for details. @listener_objects is the list of objects that should have their handles printed to by $object when Expect::interconnect, $object->expect() or $object->send_slow() are called. Calling w/out parameters will return the current list of the listener objects. Sets/unsets whether or not Expect should make reasonable guesses as to when and how to set tty parameters for $object. Will match $Expect::Manual_Stty value (normally 0) when $object is created. If called without parameters manual_stty() will return the current manual_stty setting. Set the maximum accumulator size for object. This is useful if you think that the accumulator will grow out of hand during expect() calls. Since the buffer will be matched by every match_pattern it may get slow if the buffer gets too large. Returns current value if called without parameters. Not defined by default. Return pid of $object, if one exists. Initialized filehandles will not have pids (of course). print each character from each string of @strings one at a time with $delay seconds before each character. This is handy for devices such as modems that can be annoying if you send them data too fast. After each character $object will be checked to determine whether or not it has any new data ready and if so update the accumulator for future expect() calls and print the output to STDOUT and @listen_group if log_stdout and log_group are appropriately set. Defaults to 0. Newly created objects have a $object->debug() value of $Expect::Debug. See $object->debug(); Defaults to 0. When destroying objects, soft_close may take up to half a minute to shut everything down. From now on, only hard_close will be called, which is less polite but still gives the process a chance to terminate properly. Set this to '1' for old behaviour. Defaults to 0. Newly created objects have a $object->exp_internal() value of $Expect::Exp_Internal. See $object->exp_internal(). Defaults to 1. Newly created objects have a $object->log_group() value of $Expect::Log_Group. See $object->log_group(). Defaults to 1 for spawned commands, 0 for file handles attached with exp_init(). Newly created objects have a $object->log_stdout() value of $Expect::Log_Stdout. See $object->log_stdout(). Defaults to 0. Newly created objects have a $object->manual_stty() value of $Expect::Manual_Stty. See $object->manual_stty().
        Defaults to 1. Affects whether or not expect() uses the /m flag for
doing regular expression matching. If set to 1 /m is used.
        This makes a difference when you are trying to match ^ and $. If
you have this on you can match lines in the middle of a page of output
using ^ and $ instead of it matching the beginning and end of the entire
expression. I think this is handy.

CONTRIBUTIONS
Lee Eakin <leakin@japh.itg.ti.com> has ported the kibitz script from Tcl/Expect to Perl/Expect. You can find it in the examples/ subdir. Thanks Lee!

There are still a few lines of code dating back to the inspirational Comm.pl and Chat.pl modules without which this would not have been possible. Kudos to Eric Arnold <Eric.Arnold@Sun.com> and Randal 'Nuke your NT box with one line of perl code' Schwartz<merlyn@stonehenge.com> for making these available to the perl public.

As of .98 I think all the old code is toast. No way could this have been done without it though. Special thanks to Graham Barr for helping make sense of the IO::Handle stuff as well as providing the highly recommended IO::Tty module.

REFERENCES
Mark Rogaski <rogaski@att.com> wrote:

"I figured that you'd like to know that Expect.pm has been very useful to AT&T Labs over the past couple of years (since I first talked to Austin about design decisions). We use Expect.pm for managing the switches in our network via the telnet interface, and such automation has significantly increased our reliability. So, you can honestly say that one of the largest digital networks in existence (AT&T Frame Relay) uses Expect.pm quite extensively."

FAQ - Frequently Asked Questions
This is a growing collection of things that might help. Please send you questions that are not answered here to RGiersig@cpan.org Expect itself doesn't have real system dependencies, but the underlying IO::Tty needs pseudoterminals. IO::Stty uses POSIX.pm and Fcntl.pm.

I have used it on Solaris, Linux and AIX, others report *BSD and OSF as working. Generally, any modern POSIX Unix should do, but there are exceptions to every rule. Feedback is appreciated.

See the IO::Tty manpage for a list of verified systems. Up to now, the answer was 'No', but this has changed (muchas gracias to Nick Scott <Nick.Scott@veritect.com>)!

You still cannot use ActivePerl, but if you use the Cygwin environment (http://sourceware.cygnus.com), which brings its own perl, and have the latest IO::Tty (v0.05) installed, it should work (feedback appreciated). If you set

  $Expect::Exp_Internal = 1;
Expect will tell you very verbosely what it is receiving and sending, what matching it is trying and what it found. You can do this on a per-command base with

  $exp->exp_internal(1);
You can also set

  $Expect::Debug = 1;  # or 2, 3 for more verbose output
or

  $exp->debug(1);
which gives you even more output. Yes, just set

  $Expect::Log_Stdout = 0;
to globally disable it or

   $exp->log_stdout(0);
for just that command. 'log_user' is provided as an alias so Tcl/Expect user get a DWIM experience... :-) This is caused by the pty, which has probably 'echo' enabled. A solution would be to set the pty to raw mode, which in general is cleaner for communication between two programs (no more unexpected character translations). Unfortunately this would break a lot of old code that sends "\r" to the program instead of "\n" (translating this is also handled by the pty), so I won't add this to Expect just like that. But feel free to experiment with `$exp->stty("raw")'. Try to add a 'sleep 1' after you spawn() the program. It could be that pty creation on your system is just slow. Use

  $exp->log_file("filename");
or

  $exp->log_file($filehandle);
or even

  $exp->log_file(\&log_procedure);
for maximum flexibility.

to start logging. Note that the logfile is appended to by default, but you can specify an optional mode "w" to truncate the logfile:

  $exp->log_file("filename", "w");
To stop logging, just call it with a false argument:

  $exp->log_file(undef);
To globally unset multi-line matching for all regexps:

  $Expect::Multiline_Matching = 0;
You can do that on a per-regexp basis by stating `(?-m)' inside the regexp (you need perl5.00503 or later for that). You can use the -i parameter to specify a single object or a list of Expect objects. All following patterns will be evaluated against that list.

You can specify -i multiple times to create groups of objects and patterns to match against within the same expect statement.

This works just like in Tcl/Expect.

See the source example below. Well, pty handling is really a black magic, as it is extremely system dependend. I'm thinking about stealing the pty stuff from Tcl/Expect (Hi, Don!), as that code looks much more mature than what gets done in Expect and/or IO::Tty. Please be patient, this may take a while...

If your system is listed in the "verified" list of IO::Tty, you probably have some non-standard setup, e.g. you compiled your Linux-kernel yourself and disabled ptys. Please ask your friendly sysadmin for help.

If your system is not listed, unpack the latest version of IO::Tty, do a 'perl Makefile.PL; make; make test; uname `-a'' and send me the results and I'll see what I can deduce from that. The OS may not be configured to grant additional pty's (pseudo terminals) to non-root users. /usr/sbin/mkpts should be 4755, not 700 for this to work. I don't know about security implications if you do this. By using expect without any patterns to match.

  $process->expect(undef); # Forever until EOF
  $process->expect($timeout); # For a few seconds
  $process->expect(0); # Is there anything ready on the handle now?
  $read = $process->before();
Find it on CPAN as IO::Tty, which provides both. What's happening is you are closing the handle before passwd exits. When you close the handle to a process it is sent a signal (SIGPIPE?) telling it that STDOUT has gone away. The default behavior for processes is to die in this circumstance. Two ways you can make this not happen are:

  $process->soft_close();
This will wait 15 seconds for a process to come up with an EOF by itself before killing it.

  $process->expect(undef);
This will wait forever for the process to match an empty set of patterns. It will return when the process hits an EOF. Output is only printed to the group when Expect reads from the process, during expect(), send_slow() and interconnect(). One way you can force this is to make use of

  $process->expect(undef);
and

  $process->expect(0);
which will make expect() run with an empty pattern set forever or just for an instant to capture the output of $process. The output is available in the accumulator, so you can grab it using $process->exp_before(). There seems to be a problem in IO::Tty. More to the point, HP-UX. When a process dies the parent end of it's tty does not receive an EOF, or so I have been told. I do not have a machine to test this on.

One possible solution is when you spawn a process, follow it with a unique string that would indicate the process is finished.

  $process = Expect->spawn('telnet somehost; echo ____END____');
And then $process->expect($timeout,'____END____','other','patterns'); What appears to happen is that ptys are getting reused before they are ready. This causes processes to receive premature EOFs. A workaround is to sleep() a few seconds between spawning processes. Be sure you soft_close() or hard_close() the process when you are finished.

Also, try IO::Tty v0.05, which has several fixes and should work much better on *BSD systems. Tty settings are a major pain to keep track of. If you find unexpected behavior such as double-echoing or a frozen session doublecheck the documentation for default settings. When in doubt handle them yourself using the exp_stty() and manual_stty() functions. As of .98 you shouldn't have to worry about stty settings getting fouled unless you use interconnect or intentionally change them (like doing -echo to get a password).

If you foul your terminal's tty settings, kill any hung processes and enter 'stty sane' at a shell prompt. This should make your terminal manageable again.

Note that IO::Tty returns ptys with your systems default setting regarding echoing, CRLF translation etc. and Expect does not change them. I have considered setting the ptys to raw without any translation whatsoever, but this would break a lot of existing things, as \r would not work anymore. On the other hand, a raw pty works much like a pipe and is more WYGIWYE (what you get is what you expect).

Source Examples
  my $exp = Expect->spawn("telnet localhost");
  my $spawn_ok;
  $exp->expect($timeout,
               [
                qr'login: $',
                sub {
                  $spawn_ok = 1;
                  my $fh = shift;
                  $fh->send("$username\r");
                  exp_continue;
                }
               ],
               [
                'Password: $',
                sub {
                  my $fh = shift;
                  print $fh "$password\r";
                  exp_continue;
                }
               ],
               [
                eof =>
                sub {
                  if ($spawn_ok) {
                    die "ERROR: premature EOF in login.\n";
                  } else {
                    die "ERROR: could not spawn telnet.\n";
                  }
                }
               ],
               [
                timeout =>
                sub {
                  die "No login.\n";
                }
               ],
               '-re', qr'[#>:] $', #' wait for shell prompt, then exit expect
              );
  foreach my $cmd (@list_of_commands) {
    push @commands, Expect->spawn($cmd);
  }
  expect($timeout,
         '-i', \@commands,
         [
          qr"pattern",          # find this pattern in output of all commands
          sub {
            my $obj = shift;    # object that matched
            print $obj "something\r";
            exp_continue;       # we don't want to terminate the expect call
          }
         ],
         '-i', $some_other_command,
         [
          "some other pattern",
          sub {
            my ($obj, $parmref) = @_;
            # ...
            # now we exit the expect command
          },
          \$parm
         ],
        );

HOMEPAGE
http://sourceforge.net/projects/expectperl/

MAILING LISTS
There are two mailing lists available, expectperl-announce and expectperl-discuss, at

  http://lists.sourceforge.net/lists/listinfo/expectperl-announce
and

  http://lists.sourceforge.net/lists/listinfo/expectperl-discuss

AUTHORS
(c) 1997 Austin Schutz <ASchutz@users.sourceforge.net> (retired)

expect() interface & functionality enhancements (c) 1999-2001 Roland Giersig.

This module is now maintained by Roland Giersig <RGiersig@cpan.org>

LICENSE
This module can be used under the same terms as Perl.