SEPIA: Simple Emacs Perl Integration

Sepia.jpg

Sepia is a set of Perl development tools for Emacs supporting code navigation and interactive evaluation.

1 Introduction

Sepia is a set of tools for Perl development in Emacs. Its goal is to extend CPerl mode to support fast code navigation and interactive development. It is inspired by Emacs' current support for a number of other languages, including Lisp, Python, and Emacs Lisp.

1.1 Getting Started

To install Sepia, its Emacs Lisp files must be in Emacs' load-path, and the lib directory must be in Perl's @INC. Assuming that Sepia has been unpacked in ~/sepia, it can be installed by adding the following lines to ~/.emacs:

     (add-to-list 'load-path "~/sepia")
     (setq sepia-perl5lib (list (expand-file-name "~/sepia/lib")))
     (defalias 'perl-mode 'sepia-mode)
     (require 'sepia)

Then to bring up the interactive Perl prompt, type M-x sepia-repl.

1.2 Philosophy

A development environment should support three activities: code spelunking, interaction, and customization. Emacs as an environment for developing Emacs Lisp thoroughly supports all of them: It has commands to visit individual functions' code and documentation, commands to evaluate or step through expressions, and an architecture that encourages customization in Emacs Lisp. As an environment for Perl, however, it is lacking: there is limited interactivity with the Perl debugger, and reasonable documentation browsing, but no support for navigating, editing, and re-evaluating code. Sepia attempts to remedy the situation.

Modern IDEs also support these three activities, but do so awkwardly. Rather than having functions to visit definitions (find-function) and search for functions (apropos), they clutter the screen with class and file trees. Rather than supporting interactive evaluation of small pieces of code, they perform background semantic checking on whole projects and highlight errors. Rather than allowing minor customizations to grow organically into features, they support limited configuration files and baroque plug-in APIs. Sepia tries to adhere to the apparent Emacs philosophy that rich semantic information should be unobtrusive, and that the best way to build working code is to start by experimenting with small pieces.

Language support packages for Emacs vary widely in the degree to which they make use of or replace existing Emacs features. Minimal modes provide keyword-based syntax highlighting and an unadorned comint buffer as an interpreter. Others provide their own specialized equivalents of comint, eldoc, completion, and other Emacs features. Sepia takes a third approach by trying to do as much as possible with existing Emacs features, even when they are not optimal for Perl. For example, it uses comint to communicate with the subprocess, eldoc to display documentation, and grep to list source locations.

This approach has three advantages: First, it maximizes the number of features that can be supported with limited development time. Second, it respects users' settings. A seasoned Emacs user may have changed hundreds of settings, so a mode that reimplements features will have to support equivalent settings, and will force the user to re-specify them. Finally, this approach respects decades of development spent, as Neal Stephenson put it, “focused with maniacal intensity on the deceptively simple-seeming problem of editing text.” Many non-obvious choices go into making a polished interface, and while a reimplementation gets rid of accumulated cruft, it must rediscover these hidden trade-offs.

Anyways, I hope you enjoy using Sepia. Its development style is strange for someone used Perl's typical mix of one-liners and edit-save-run, but once you are accustomed to it, you may find it very effective.

1.3 Related Work

A number of more-or-less related Emacs extensions are currently under development. Here is a list of the ones I have heard about, along with my brief impression of how they differ from Sepia. Since I use none of them regularly, these impressions should be taken with a grain of salt.

PDE
PDE is similar to Sepia in offering an interactive Lisp-like development environment interfacing with a long-running Perl process. It seems more ambitious, and therefore a bit more invasive.

http://search.cpan.org/search?query=pde&mode=module

Devel::PerlySense
Devel::PerlySense offers a more Eclipse-like development environment, with offline code analysis via PPI.

http://search.cpan.org/search?query=devel::perlysense&mode=module

Emacs::EPL
Emacs::EPL is a low-level IPC interface between Emacs and Perl. Sepia was originally based on Emacs::EPL, but the current comint-based implementation proved more maintainable.

http://search.cpan.org/search?query=emacs::epl&mode=module

2 Editing

Sepia's first contribution is a set of commands to explore a Perl codebase. These include commands to browse and display documentation, to find function definitions, and to query a cross-reference database of function and variable uses. Sepia also provides intelligent symbol completion.

2.1 Completion

Sepia implements partial-word completion that communicates with the inferior Perl process. For example, `%S:X:v_u' completes to `%Sepia::Xref::var_use' when Sepia is loaded. This completion only operates on functions and global variables known to the Perl interpreter, so it works best when code and interpreter are in sync.

More precisely, completion examines the text before point and tries each of the following in turn, using the first successful approach:

  1. If the text looks like a method call (e.g. `$object->f' or `Class->f'), complete on methods.
  2. If it looks like a variable (e.g. `%hash' or `$scalar'), complete on variables.
  3. Complete on modules and functions.
  4. Otherwise, complete on Perl built-in operators.

For each of the first three cases, completions candidates are first generated by splitting the text on characters [:_] and matching the resulting word parts. For example, `X:a_b' will complete to all symbols matching `^X[^:]*:+a[^:_]*_b' such as `Xref::a_bug' and `X::always_bites_me'. If no matches result, the text is treated as an acronym. For example, `dry' will complete to `dont_repeat_yourself'.

Completion is performed by the following commands:

M-x sepia-complete-symbol
Complete the symbol before point as described above. Note that this does not consider lexical scope, and is always case-sensitive, independent of completion-ignore-case.
TAB
M-x sepia-indent-or-complete
First try to reindent the current line. If its indentation does not change, then try to expand an abbrev at point (unless sepia-indent-expand-abbrev is nil). If no abbrev is expanded, then call sepia-complete-symbol.

2.2 Navigation

Sepia provides several commands for navigating program source. All of them rely on information from the inferior Perl process, so it is important both that it be running, and that its internal representation of the program match the program source. The commands marked (Xref) below also rely on a cross-reference database, which must be explicitly rebuilt by calling xref-rebuild when the program changes.

There are two basic kinds of navigation commands. The first kind jumps directly to the first matching location when possible, prompting only if no such location is found. These commands find only a single location.

M-. M-.
M-x sepia-dwim
Guess what kind of identifier is at point, and try to do the right thing: for a function, find its definition(s); for a variable, find its uses; for a module, view its documentation; otherwise, prompt for the name of a function to visit. sepia-dwim automatically goes to the first function definition or variable use found.
M-. l
M-x sepia-location
Jump directly to the definition of the function at point, prompting if point is not on a known function. If multiple definitions are found, choose one arbitrarily. This function is similar to sepia-defs, and the two should probably be merged.
M-. j
M-x sepia-jump-to-symbol
Navigate to a function using “ido” interactive completion. Within interactive completion, press <:> to descend into a package, <DEL> to ascend to a parent package, and <RET> to go to the currently-selected function.

The second kind of navigation commands always prompts the user – though usually with a sensible default value – and finds multiple locations. When called with a prefix argument, these commands present their results in a grep-mode buffer. When called without a prefix argument, they place all results on the found-location ring and jump directly to the first. The remaining locations can be cycled through by calls to sepia-next.

M-. f name <RET>
M-x sepia-defs
Find definition(s) of function name.
M-. m name <RET>
M-x sepia-module-find name <RET>
Find the source of module name.
M-. a regexp <RET>
M-x sepia-apropos regexp <RET>
Find definitions of all functions whose names match regexp.
M-. c name <RET>
M-x sepia-callers name <RET>
(Xref) Find calls to function name.
M-. C name <RET>
M-x sepia-callees name <RET>
(Xref) Find the definitions of functions called by name.
M-. v name <RET>
M-x sepia-var-uses name <RET>
(Xref) Find uses of the global variable name.
M-. V name <RET>
M-x sepia-var-defs name <RET>
(Xref) Find definitions of global variable name. Since Perl's global variables are not declared, this is rarely useful

Finally, there are several other navigation-related commands that do not fit into either of the above categories.

M-,
M-x sepia-next
Cycle through the definitions found by the previous <M-.> search.
M-. r
M-x sepia-rebuild
Rebuild the cross-reference database by walking the op-tree and stashes.
M-. t
M-x find-tag
Execute the find-tag command typically bound to <M-.>.

2.3 Documentation

Sepia can be used to browse installed modules' documentation, to format and display the current buffer's POD, and to browse the list of modules installed on the system.

M-. d name <RET>
M-x sepia-perldoc-this
View documentation for module name or Perl manual page name.
C-c C-d
M-x sepia-view-pod
Format and view the current buffer's documentation.
sepia-package-list
Browse a tree of installed packages. This lists only the top-level packages from installed distributions, so if package My::Stuff also provides My::Stuff::Details, it will not be displayed. When Emacs-w3m is available, each module is linked to its documentation.
sepia-module-list
Browse a tree of both top-level and internal packages, like sepia-package-list.

Sepia also integrates with eldoc (at least in GNU Emacs >= 22). Documentation for Perl operators and control structures is taken from CPerl mode. Sepia will also display documentation for user-defined functions if their POD is formatted in the standard way, with functions described in a “=head2” or “=item” entry. To load user documentation, visit the relevant file and type M-x sepia-doc-update.

If Module::CoreList is available, Sepia's eldoc function will also display the first version of Perl with which a module was shipped. This is intended to give the programmer a sense of when he is creating external dependencies.

3 Interactive Perl

Sepia's second main contribution is an interactive interface (REPL) to an inferior Perl process. The interface is based on GUD mode, and inherits many of its bindings; this chapter discusses only the Sepia extensions. To start or switch to the repl, type M-x sepia-repl. As in Sepia mode, <TAB> in the REPL performs partial-word completion with sepia-complete-symbol.

Sepia also provides a number of other ways to evaluate pieces of code in Perl, and commands to process buffer text using the inferior process.

3.1 Shortcuts

“Shortcuts” are commands handled specially by the REPL rather than being evaluated as Perl code. They either communicate with the REPL function, or provide a convenient interface to variables in the Sepia package. Shortcuts are prefixed by a comma (<,>), and may be abbreviated to the shortest unique prefix.

cd dir
Change Perl's current directory to dir.
debug [val]
Turn Sepia debugger hook on or off, or toggle if val is missing.
define name ['doc'] body...
Define name as a shortcut for Perl code body, with optional documentation doc, surrounded by single quotes. body is passed the raw command-line text as its first argument.
delete
Delete the current breakpoint.
format type
Set the output format to type, either “dumper” (using Data::Dumper), “dump” (Data::Dump), “yaml” (YAML), or “plain” (stringification). Default: “dumper”.
help
Display a list of shortcuts.
lsbreak
List breakpoints.
methods name [regexp]
Display a list of functions defined in package name and its ISA-ancestors matching optional pattern regexp.
package name
Set the default evaluation package to name.
pwd
Show the process's current working directory.
quit
Exit the inferior Perl process.
reload
Reload Sepia.pm and recursively invoke the REPL. This command is mostly of interest when working on Sepia itself.
shell [command]
Execute shell command command, displaying its standard output and standard error.
strict [val]
Set evaluation strictness to val, or toggle it if val is not given. Note that turning strictness off and on clears the REPL's lexical environment.
undef name
Undefine shortcut name. Warning: this can equally be used to remove built-in shortcuts.
wantarray [val]
Set the evaluation context to val, or toggle between scalar and array context.
who package [regexp]
who [regexp]
List identifiers in package (main by default) matching optional regexp.

3.2 Debugger

Sepia uses Perl's debugger hooks and GUD mode to support conditional breakpoints and single-stepping, and overrides Perl's die() to invoke the debugger rather than unwind the stack. This makes it possible to produce a backtrace, inspect and modify global variables, and even continue execution when a program tries to kill itself. If the PadWalker module is available, Sepia also provides functions to inspect and modify lexical variables.

The debugger has its own set of shortcuts, also prefixed by a comma.

backtrace
Show a backtrace.
break file:line [expr]
Set a breakpoint in file at line. If expr is supplied, stop only if it evaluates to true.
down n
up n
Move the current stack frame up or down by n (or one) frames.
inspect [n]
Inspect lexicals in the current frame or frame n, counting upward from 1.
lsbreak
List breakpoints.
next [n]
Advance n (or one) lines, skipping subroutine calls.
quit
die
warn
Continue as the program would have executed without debugger intervention, dying if the debugger was called from die().
return expr
Continue execution as if die() had returned the value of expr, which is evaluated in the global environment.
step [n]
Step forward n (or one) lines, descending into subroutines.
xreturn sub expr
Return expr from the innermost call to sub. This is a somewhat dangerous and experimental feature, but is probably more useful than returning a value from die().

3.3 Evaluation

When interactive Perl is running, Sepia can evaluate regions of code in the inferior Perl process. The following commands assume that this process has already been started by calling sepia-repl.

C-M-x
M-x sepia-eval-defun
Evaluate the function around point in the inferior Perl process. If it contains errors, jump to the location of the first.
C-c C-l
M-x sepia-load-file
Save the current buffer, then reload its file and if warnings or errors occur, display an error buffer. With a prefix argument, also rebuild the cross-reference index.
C-c e
M-x sepia-eval-expression <RET> expr <RET>
Evaluate expr in scalar context and echo the result. With a prefix argument, evaluate in list context.
C-c!
sepia-set-cwd
Set the REPL's working directory to the current buffer's directory.

3.4 Mutilation

Sepia contains several functions to operate on regions of text using the interactive Perl process. These functions can be used like standard one-liners (e.g. `perl -pe ...'), with the advantage that all of the functions and variables in the interactive session are available.

M-x sepia-perl-pe-region <RET> code <RET>
Evaluate code on each line in the region with $_ bound to the line text, collecting the resulting values of $_. With a prefix argument, replace the region with the result.
M-x sepia-perl-ne-region <RET> code <RET>
Evaluate code as above, but collect the results instead.
M-x sepia-perlize-region <RET> code <RET>
Evaluate code once with $_ bound to the entire region, collecting the final value of $_. With a prefix argument, replace the region.

3.5 Scratchpad

Sepia also supports a scratchpad, another form of interaction inspired by Emacs' *scratch* buffer. To create or switch to the scratchpad, type M-x sepia-scratch. Scratchpad mode is exactly like Sepia mode, except <C-j> evaluates the current line and prints the result on the next line.

4 CPAN browsing

Sepia has rudimentary support for browsing documentation and installing modules from CPAN. Modules whose names, descriptions, or authors match a query are displayed in a *sepia-cpan* buffer, in which the following commands are available:

s
M-x sepia-cpan-search <RET> pattern <RET>
List modules whose names match pattern.
/
M-x sepia-cpan-desc <RET> pattern <RET>
List modules whose names or descriptions match pattern.
l
M-x sepia-cpan-list <RET> name <RET>
List modules authored by name.
r
M-x sepia-cpan-readme <RET> module <RET>
Fetch and display module's README file.
d
M-x sepia-cpan-doc <RET> module <RET>
Browse module's documentation on http://search.cpan.org.
i
M-x sepia-cpan-install <RET> module <RET>
Install module and its prerequisites. This feature is not yet well tested.

5 Customization

While Sepia can be customized in both the Perl and Emacs Lisp, most of the user-accessible configuration is in the latter.

5.1 Emacs Variables

Since Sepia tries where possible to reuse existing Emacs functionality, its behavior should already be covered by existing customizations. The two variables most likely to need customization are sepia-program-name and sepia-perl5lib. General Sepia mode configuration can be done with sepia-mode-hook, while REPL-specific configuration can be done with sepia-repl-mode-hook.

sepia-complete-methods
If non-nil, sepia-complete-symbol will complete simple method calls of the form $x-> or Module->. Since the former requires evaluation of $x, this can be disabled. Default: T.
sepia-eval-defun-include-decls
If non-nil, attempt to generate a declaration list for sepia-eval-defun. This is necessary when evaluating some code, such as that calling functions without parentheses, because the presence of declarations affects the parsing of barewords. Default: T.
sepia-indent-expand-abbrev
If non-nil, sepia-indent-or-complete will, if reindentation does not change the current line, expand an abbreviation before point rather than performing completion. Only if no abbreviation is found will it perform completion. Default: T.
sepia-module-list-function
The function to view a tree of installed modules. Default: w3m-find-file if Emacs-w3m is installed, or browse-url-of-buffer otherwise.
sepia-perldoc-function
The function called to view installed modules' documentation. Default: w3m-perldoc if Emacs-w3m is installed, or cperl-perldoc otherwise.
sepia-perl5lib
A list of directories to include in PERL5LIB when starting interactive Perl. Default: nil.
sepia-prefix-key
The prefix to use for for functions in sepia-keymap. Default: <M-.>.
sepia-program-name
The Perl program name for interactive Perl. Default: “perl”.
sepia-use-completion
If non-nil, various Sepia functions will generate completion candidates from interactive Perl when called interactively. This may be slow or undesirable in some situations. Default: T.
sepia-view-pod-function
The function called to view the current buffer's documentation. Default: sepia-w3m-view-pod if Emacs-w3m is available, or sepia-perldoc-buffer otherwise.

5.2 Perl Variables

When Sepia starts up, it evaluates the Perl script in ~/.sepiarc. The following variables in the Sepia package control various aspects of interactive evaluation.

$PACKAGE
The package in which user input is evaluated, determined automatically when code is evaluated from a buffer. Default: main.
$PRINTER
The function called to format interactive output, normally set with the printer shortcut.
$PRINT_PRETTY
If true, format some values nicely independent of the value of $PRINTER. Currently, this means columnating lists of simple scalars. Default: true.
$PS1
The trailing end of the prompt string, which should end with “> ”. Default: "> ".
$STOPDIE
If true, calls to die from interactive code will invoke the Sepia debugger. Default: true.
$STOPWARN
If true, calls to warn from interactive code will invoke the Sepia debugger. Default: false.
$WANTARRAY
If true, evaluate interactive expressions in list context. Default: true.

Additional REPL shortcuts can be defined with Sepia::define_shortcut. For example

     Sepia::define_shortcut time => sub { print scalar localtime, "\n"; 0 },
         'Display the current time.';

defines a shortcut “time” that displays the current time. For details, see the code in Sepia.pm.

6 Internals

Many things remain unexplained except by the code itself, and some details mentioned above should probably be given less prominence. For developer documentation, please see the POD for Sepia and Sepia::Xref, and the doc-strings in sepia.el.

Credits

Hilko Bengen
Found and motivated me to fix a bunch of bugs, created Debian packages.
Ævar Arnfjörð Bjarmason
Miscellaneous fixes. Tested unicode support.
Ye Wenbin
Found and fixed numerous bugs.
Free Software
Portions of the code were lifted from Emacs-w3m, SLIME, ido, and B::Xref, all of which are Free software.

Function Index