devel/sed/sedfaq.txt


Archive-Name: editor-faq/sed
Posting-Frequency: irregular
Last-modified: 10 March 2003
Version: 015
URL: http://sed.sourceforge.net/sedfaq.html
Maintainer: Eric Pement (pemente@northpark.edu)

                            THE SED FAQ

                  Frequently Asked Questions about
                       sed, the stream editor

CONTENTS

1. GENERAL INFORMATION
1.1. Introduction - How this FAQ is organized
1.2. Latest version of the sed FAQ
1.3. FAQ revision information
1.4. How do I add a question/answer to the sed FAQ?
1.5. FAQ abbreviations
1.6. Credits and acknowledgements
1.7. Standard disclaimers

2. BASIC SED
2.1. What is sed?
2.2. What versions of sed are there, and where can I get them?

2.2.1. Free versions

2.2.1.1. Unix platforms
2.2.1.2. OS/2
2.2.1.3. Microsoft Windows (Win3x, Win9x, WinNT, Win2K)
2.2.1.4. MS-DOS
2.2.1.5. CP/M
2.2.1.6. Macintosh v8 or v9

2.2.2. Shareware and Commercial versions

2.2.2.1. Unix platforms
2.2.2.2. OS/2
2.2.2.3. Windows 95/98, Windows NT, Windows 2000
2.2.2.4. MS-DOS

2.3. Where can I learn to use sed?

2.3.1. Books
2.3.2. Mailing list
2.3.3. Tutorials, electronic text
2.3.4. General web and ftp sites

3. TECHNICAL
3.1. More detailed explanation of basic sed
3.1.1.  Regular expressions on the left side of "s///"
3.1.2.  Escape characters on the right side of "s///"
3.1.3.  Substitution switches
3.2. Common one-line sed scripts. How do I . . . ?

      - double/triple-space a file?
      - convert DOS/Unix newlines?
      - delete leading/trailing spaces?
      - do substitutions on all/certain lines?
      - delete consecutive blank lines?
      - delete blank lines at the top/end of the file?

3.3. Addressing and address ranges
3.4. Address ranges in GNU sed and HHsed
3.5. Debugging sed scripts
3.6. Notes about s2p, the sed-to-perl translator
3.7. GNU/POSIX extensions to regular expressions

4. EXAMPLES
   ONE-CHARACTER QUESTIONS
4.1.  How do I insert a newline into the RHS of a substitution?
4.2.  How do I represent control-codes or non-printable characters?
4.3.  How do I convert files with toggle characters, like +this+,
      to look like [i]this[/i]?

   CHANGING STRINGS
4.10. How do I perform a case-insensitive search?
4.11. How do I match only the first occurrence of a pattern?
4.12. How do I parse a comma-delimited (CSV) data file?
4.13. How do I handle fixed-length, columnar data?
4.14. How do I commify a string of numbers?
4.15. How do I prevent regex expansion on substitutions?
4.16. How do I convert a string to all lowercase or capital letters?

   CHANGING BLOCKS (consecutive lines)
4.20. How do I change only one section of a file?
4.21. How do I delete or change a block of text if the block contains
      a certain regular expression?
4.22. How do I locate a paragraph of text if the paragraph contains a
      certain regular expression?
4.23. How do I match a block of specific consecutive lines?
4.23.1.  Try to use a "/range/, /expression/"
4.23.2.  Try to use a "multi-line\nexpression"
4.23.3.  Try to use a block of "literal strings"
4.24. How do I address all the lines between RE1 and RE2, excluding the lines themselves?
4.25. How do I join two lines if line #1 ends in a [certain string]?
4.26. How do I join two lines if line #2 begins in a [certain string]?
4.27. How do I change all paragraphs to long lines?

   SHELL AND ENVIRONMENT
4.30.   How do I read environment variables with sed ...
4.31.1.   ... on Unix platforms?
4.31.2.   ... on MS-DOS or 4DOS platforms?
4.32.   How do I export or pass variables back into the environment ...
4.32.1.   ... on Unix platforms?
4.32.2.   ... on MS-DOS or 4DOS platforms?
4.33.   How do I handle shell quoting in sed?

   FILES, DIRECTORIES, AND PATHS
4.40.  How do I read (insert/add) a file at the top of a textfile?
4.41.  How do I make substitutions in every file in a directory, or
        in a complete directory tree?
4.41.1.   ... ssed solution
4.41.2.   ... Unix solution
4.41.3.   ... DOS solution
4.42.  How do I replace "/some/UNIX/path" in a substitution?
4.43.  How do I replace "C:\SOME\DOS\PATH" in a substitution?
4.44.  How do I emulate file-includes, using sed?

5. WHY ISN'T THIS WORKING?
5.1.  Why don't my variables like $var get expanded in my sed script?
5.2.  I'm using 'p' to print, but I have duplicate lines sometimes.
5.3.  Why does my DOS version of sed process a file part-way through
      and then quit?
5.4.  My RE isn't matching/deleting what I want it to. (Or, "Greedy vs.
      stingy pattern matching")
5.5.  What is CSDPMI*B.ZIP and why do I need it?
5.6.  Where are the man pages for GNU sed?
5.7.  How do I tell what version of sed I am using?
5.8.  Does sed issue an exit code?
5.9.  The 'r' command isn't inserting the file into the text.
5.10. Why can't I match or delete a newline using the \n escape
      sequence? Why can't I match 2 or more lines using \n?
5.11. My script aborts with an error message, "event not found".

6. OTHER ISSUES
6.1.  I have a problem that stumps me. Where can I get help?
6.2.  How does sed compare with awk, perl, and other utilities?
6.3.  When should I use sed?
6.4.  When should I NOT use sed?
6.5.  When should I ignore sed and use Awk or Perl instead?
6.6.  Known limitations among sed versions
6.7.  Known incompatibilities between sed versions

6.7.1. Issuing commands from the command line
6.7.2. Using comments (prefixed by the '#' sign)
6.7.3. Special syntax in REs
6.7.4. Word boundaries
6.7.5. Commands which operate differently

7. KNOWN BUGS AMONG SED VERSIONS
7.1. ssed v3.59
7.2. GNU sed v4.0 - v4.0.5
7.3. GNU sed v3.02.80
7.4. GNU sed v3.02
7.5. GNU sed v2.05
7.6. GNU sed v1.18
7.7. GNU sed v1.03
7.8. sed v1.6 (Briscoe)
7.9. sed v1.5 (Helman)
7.10. sedmod v1.0 (Chen)
7.11. HP-UX sed
7.12. SunOS sed v4.1
7.13. SunOS sed v5.6
7.14. Ultrix sed v4.3
7.15. Digital Unix sed


------------------------------

1. GENERAL INFORMATION

1.1. Introduction - How this FAQ is organized

   This FAQ is organized to answer common (and some uncommon)
   questions about sed, quickly. If you see a term or abbreviation in
   the examples that seems unclear, see if the term is defined in
   section 1.5. If not, send your comment to pemente[at]northpark.edu.

1.2. Latest version of the sed FAQ

   The newest version of the sed FAQ is usually here:

       http://sed.sourceforge.net/sedfaq.html (HTML version)
       http://sed.sourceforge.net/sedfaq.txt  (plain text)
       http://www.student.northpark.edu/pemente/sed/sedfaq.html
       http://www.student.northpark.edu/pemente/sed/sedfaq.txt
       http://www.faqs.org/faqs/editor-faq/sed
       ftp://rtfm.mit.edu/pub/faqs/editor-faq/sed

   Another FAQ file on sed by a different author can be found here:

       http://www.dreamwvr.com/sed-info/sed-faq.html

1.3. FAQ revision information

   In the plaintext version, changes are shown by a vertical bar (|)
   placed in column 78 of the affected lines. To remove the vertical
   bars (use double quotes for MS-DOS):

     sed 's/  *|$//' sedfaq.txt > sedfaq2.txt

   In the HTML version, vertical bars do not appear. New or altered
   portions of the FAQ are indicated by printing in dark blue type.

   In the text version, words needing emphasis may be surrounded by
   the underscore '_' or the asterisk '*'. In the HTML version, these
   are changed to italics and boldface, respectively.

1.4. How do I add a question/answer to the sed FAQ?

   Word your question briefly and send it to pemente[at]northpark.edu,
   indicating your proposed change. We'll post it on the sed-users
   mailing list (see section 2.3.2) and discuss it there. If it's
   good, your contribution will be added to the next edition.

1.5. FAQ abbreviations

       files = one or more filenames, separated by whitespace
       gsed  = GNU sed
       ssed  = super-sed
       RE    = Regular Expressions supported by sed
       LHS   = the left-hand side ("find" part) of "s/find/repl/" command
       RHS   = the right-hand side ("replace" part) of "s/find/repl/" cmd
       nn+   = version _nn_ or higher (e.g., "15+" = version 1.5 and above)

   files: "files" stands for one or more filenames entered on the
   command line. The names may include any wildcards your shell
   understands (such as ``zork*'' or ``Aug[4-9].let''). Sed will
   process each filename passed to it by the shell.

   RE: For details on regular expressions, see section 3.1.1., below.

1.6. Credits and acknowledgements

   Many of the ideas for this FAQ were taken from the Awk FAQ:
       http://www.faqs.org/faqs/computer-lang/awk/faq/
       ftp://rtfm.mit.edu/pub/usenet/comp.lang.awk/faq

   and from the old Perl FAQ:
       http://www.perl.com/doc/FAQs/FAQ/oldfaq-html/index.html

   The following individuals have contributed significantly to this
   document, and have provided input and wording suggestions for
   questions, answers, and script examples. Credit goes to these
   contributors (in alphabetical order by last name):

   Al Aab, Yiorgos Adamopoulos, Paolo Bonzini, Walter Briscoe, Jim
   Dennis, Carlos Duarte, Otavio Exel, Sven Guckes, Aurelio Jargas,
   Mark Katz, Toby Kelsey, Eric Pement, Greg Pfeiffer, Ken Pizzini,
   Niall Smart, Simon Taylor, Peter Tillier, Greg Ubben, Laurent
   Vogel.

1.7. Standard disclaimers

   While a serious attempt has been made to ensure the accuracy of the
   information presented herein, the contributors and maintainers of
   this document do not claim the absence of errors and make no
   warranties on the information provided. If you notice any mistakes,
   please let us know so we can fix it.

------------------------------

2. BASIC SED

2.1. What is sed?

   "sed" stands for Stream EDitor. Sed is a non-interactive editor,
   written by the late Lee E. McMahon in 1973 or 1974. A brief history
   of sed's origins may be found in an early history of the Unix
   tools, at <http://www.columbia.edu/~rh120/ch106.x09>.

   Instead of altering a file interactively by moving the cursor on
   the screen (as with a word processor), the user sends a script of
   editing instructions to sed, plus the name of the file to edit (or
   the text to be edited may come as output from a pipe). In this
   sense, sed works like a filter -- deleting, inserting and changing
   characters, words, and lines of text. Its range of activity goes
   from small, simple changes to very complex ones.

   Sed reads its input from stdin (Unix shorthand for "standard
   input," i.e., the console) or from files (or both), and sends the
   results to stdout ("standard output," normally the console or
   screen). Most people use sed first for its substitution features.
   Sed is often used as a find-and-replace tool.

     sed 's/Glenn/Harold/g' oldfile >newfile

   will replace every occurrence of "Glenn" with the word "Harold",
   wherever it occurs in the file. The "find" portion is a regular
   expression ("RE"), which can be a simple word or may contain
   special characters to allow greater flexibility (for example, to
   prevent "Glenn" from also matching "Glennon").

   My very first use of sed was to add 8 spaces to the left side of a
   file, so when I printed it, the printing wouldn't begin at the
   absolute left edge of a piece of paper.

     sed 's/^/        /' myfile >newfile   # my first sed script
     sed 's/^/        /' myfile | lp       # my next sed script

   Then I learned that sed could display only one paragraph of a file,
   beginning at the phrase "and where it came" and ending at the
   phrase "for all people". My script looked like this:

     sed -n '/and where it came/,/for all people/p' myfile

   Sed's normal behavior is to print (i.e., display or show on screen)
   the entire file, including the parts that haven't been altered,
   unless you use the -n switch. The "-n" stands for "no output". This
   switch is almost always used in conjunction with a 'p' command
   somewhere, which says to print only the sections of the file that
   have been specified. The -n switch with the 'p' command allow for
   parts of a file to be printed (i.e., sent to the console).

   Next, I found that sed could show me only (say) lines 12-18 of a
   file and not show me the rest. This was very handy when I needed to
   review only part of a long file and I didn't want to alter it.

     # the 'p' stands for print
     sed -n 12,18p myfile

   Likewise, sed could show me everything else BUT those particular
   lines, without physically changing the file on the disk:

     # the 'd' stands for delete
     sed 12,18d myfile

   Sed could also double-space my single-spaced file when it came time
   to print it:

     sed G myfile >newfile

   If you have many editing commands (for deleting, adding,
   substituting, etc.) which might take up several lines, those
   commands can be put into a separate file and all of the commands in
   the file applied to file being edited:

     #  'script.sed' is the file of commands
     # 'myfile' is the file being changed
     sed -f script.sed myfile  # 'script.sed' is the file of commands

   It is not our intention to convert this FAQ file into a full-blown
   sed tutorial (for good tutorials, see section 2.3). Rather, we hope
   this gives the complete novice a few ideas of how sed can be used.

2.2. What versions of sed are there, and where can I get them?

2.2.1. Free versions

   Note: "Free" does not mean "public domain" nor does it necessarily
   mean you will never be charged for it. All versions of sed in this
   section except the CP/M versions are based on the GNU general
   public license and are "free software" by that standard (for
   details, see http://www.gnu.org/philosophy/free-sw.html). This
   means you can get the source code and develop it further.

   At the URLs listed in this category, sed binaries or source code
   can be downloaded and used without fees or license payments.

2.2.1.1. Unix platforms

   ssed v3.60
   ssed is the version recommended by the FAQ maintainers, since it
   shares the same codebase with GNU sed, has the most options, and is
   free software (you can get the source). Though there were earlier
   version of ssed distributed, sites for these are not being listed.

       http://sed.sourceforge.net/grabbag/ssed
       http://freshmeat.net/project/sed/

   GNU sed v4.0.5
   This is the latest official version of GNU sed. It offers in-place
   text replacement as an option switch.

       ftp://ftp.gnu.org/pub/gnu/sed/sed-4.0.5.tar.gz
       http://freshmeat.net/project/sed

   BSD multi-byte sed (Japanese)
   Based on the latest version of GNU sed, which supports multi-byte
   characters.

       ftp://ftp1.freebsd.org/pub/FreeBSD/FreeBSD-stable/packages/Latest/ja-sed.tgz

   GNU sed v3.02.80
   An alpha test release which was the base for the development of
   ssed and GNU sed v4.0.

       ftp://alpha.gnu.org/pub/gnu/sed/sed-3.02.80.tar.gz

   GNU sed v3.02a
   Interim version with most features of GNU sed v3.02.80.

   GNU sed v3.02
       ftp://ftp.gnu.org/pub/gnu/sed/sed-3.02.tar.gz

   Precompiled versions:

   GNU sed v3.02-8
   source code and binaries for Debian GNU/Linux

       http://www.debian.org/Packages/stable/base/sed.html

   For some time, the GNU project <http://www.gnu.org> used Eric S.
   Raymond's version of sed (ESR sed v1.1), but eventually dropped it
   because it had too many built-in limits. In 1991 Howard Helman
   modified the GNU/ESR sed and produced a flexible version of sed
   v1.5 available at several sites (Helman's version permitted things
   like \<...\> to delimit word boundaries, \xHH to enter hex code and
   \n to indicate newlines in the replace string). This version did
   not catch on with the GNU project and their version of sed has
   moved in a similar but different direction.

   sed v1.3, by Eric Steven Raymond (released 4 June 1998)
       http://catb.org/~esr/sed-1.3.tar.gz

   Eric Raymond <esr@snark.thyrsus.com> wrote one of the earliest
   versions of sed. On his website <http://www.catb.org/~esr/> which
   also distributes many freeware utilities he has written or worked
   on, he describes sed v1.1 this way:

   "This is the fast, small sed originally distributed in the GNU
   toolkit and still distributed with Minix. The GNU people ditched it
   when they built their own sed around an enhanced regex package --
   but it's still better for some uses (in particular, faster and less
   memory-intensive)." (Version 1.3 fixes an unidentified bug and adds
   the L command to hexdump the current pattern space.)

2.2.1.2. OS/2

   GNU sed v3.02.80
       http://www2s.biglobe.ne.jp/~vtgf3mpr/gnu/sed.htm

   GNU sed v3.02
       http://hobbes.nmsu.edu/pub/os2/util/file/sed-3_02-r2-bin.zip # binaries
       http://hobbes.nmsu.edu/pub/os2/util/file/sed-3_02-r2.zip     # source

2.2.1.3. Microsoft Windows (Win3x, Win9x, WinNT, Win2K)

   GNU sed v4.0.5
   32-bit binaries and docs. Precompiled versions not available (yet).

   GNU sed v3.02.80
   32-bit binaries and docs, using DJGPP compiler. For details on new
   features, see Unix section, above.

       http://www.student.northpark.edu/pemente/sed/sed3028a.zip # DOS binaries
       ftp://alpha.gnu.org/pub/gnu/sed/sed-3.02.80.tar.gz        # source
       ftp://ftp.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/sed3028b.zip # binaries
       ftp://ftp.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/sed3028d.zip # docs
       ftp://ftp.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/sed3028s.zip # source

   GNU sed v2.05
   32-bit binaries, no docs. Requires 80386 DX (SX will not run) and
   must be run in a DOS window or in a full screen DOS session under
   Microsoft Windows. Will not run in MS-DOS mode (outside Win/Win95).
   We recommend using the latest version of GNU sed.
       http://www.simtel.net/pub/win95/prog/gsed205b.zip
       ftp://ftp.cdrom.com/pub/simtelnet/win95/prog/gsed205b.zip

   GNU sed v1.03
   modified by Frank Whaley.

   This version was part of the "Virtually UN*X" toolset, hosted by
   itribe.net; that website is now closed. Gsed v1.03 supported Win9x
   long filenames, as well as hex, decimal, binary, and octal
   character representations.

   The Cygwin toolkit:
       http://www.cygwin.com

   Formerly know as "GNU-Win32 tools." According to their home page,
   "The Cygwin tools are Win32 ports of the popular GNU development
   tools for Windows NT, 95 and 98. They function through the use of
   the Cygwin library which provides a UNIX-like API on top of the
   Win32 API." The version of sed used is GNU sed v3.02.

   Minimalist GNU for Windows (MinGW):
       http://www.mingw.org
       http://mingw.sourceforge.net

   According to their home page, "MinGW ('Minimalist GNU for Windows')
   refers to a set of runtime headers, used in building a compiler
   system based on the GNU GCC and binutils projects. It compiles and
   links code to be run on Win32 platforms ... MinGW uses Microsoft
   runtime libraries, distributed with the Windows operating system."
   The version of sed used is GNU sed v3.02.

   sed v1.5 (a/k/a HHsed), by Howard Helman
   Compiled with Mingw32 for 32-bit environments described above. This
   version should support Win95 long filenames.
       http://www.dbnet.ece.ntua.gr/~george/sed/OLD/sed15.exe
       http://www.student.northpark.edu/pemente/sed/sed15exe.zip

2.2.1.4. MS-DOS

   sed v1.6 (from HHsed), by Walter Briscoe

   This is a forthcoming version, now in beta testing, but with many
   new features. It corrects all the bugs in sed v1.5, and adds the
   best features of sedmod v1.0 (below). It is available in 16-bit and
   32-bit compiled versions for MS-DOS. Sorry, no URLs available yet.

   sed v1.5 (a/k/a HHsed), by Howard Helman
   uncompiled source code (Turbo C)
       ftp://ftp.simtel.net/pub/simtelnet/msdos/txtutl/sed15.zip
       ftp://ftp.cdrom.com/pub/simtelnet/msdos/txtutl/sed15.zip

   DOS executable and documentation
       ftp://ftp.simtel.net/pub/simtelnet/msdos/txtutl/sed15x.zip
       ftp://ftp.cdrom.com/pub/simtelnet/msdos/txtutl/sed15x.zip

   sedmod v1.0, by Hern Chen
       http://www.ptug.org/sed/SEDMOD10.ZIP
       http://www.student.northpark.edu/pemente/sed/sedmod10.zip
       ftp://garbo.uwasa.fi/pc/unix/sedmod10.zip

   GNU sed v3.02.80
   See section 2.2.1.3 ("Microsoft Windows"), above.

   GNU sed v2.05
   Does not run under MS-DOS.

   GNU sed v1.18
   32-bit binaries and source, using DJGPP compiler. Requires 80386 SX
   or better. Also requires 3 CWS*.EXE extenders on the path. See
   section 5.5 ("What is CSDPMI*B.ZIP and why do I need it?"), below.
   We recommend using a newer version of GNU sed.
       http://www.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/sed118b.zip
       ftp://ftp.cdrom.com/pub/simtelnet/gnu/djgpp/v2gnu/sed118b.zip
       http://www.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/sed118s.zip
       ftp://ftp.cdrom.com/pub/simtelnet/gnu/djgpp/v2gnu/sed118s.zip

   GNU sed v1.06
   16-bit binaries and source. Should run under any MS-DOS system.
       http://www.simtel.net/pub/gnu/gnuish/sed106.zip
       ftp://ftp.cdrom.com/pub/simtelnet/gnu/gnuish/sed106.zip

2.2.1.5. CP/M

   ssed v2.2, by Chuck A. Forsberg

   Written for CP/M, ssed (for "small/stupid stream editor) supports
   only the a(ppend), c(hange), d(elete) and i(nsert) options, and
   apparently doesn't support regular expressions. A -u switch will
   "unsqueeze" compressed files and was used mainly in conjunction
   with DIF.COM for source code maintenance. (file: ssed22.lbr)

   change, by Michael M. Rubenstein

   Rubenstein released a version of sed called CHANGE.COM (the
   TTOOLS.LBR archive member CHANGE.CZM is a "crunched" file).
   CHANGE.COM supports full RE's except grouping and backreferences,
   and its only function is global substitution. (file: ttools.lbr)

2.2.1.6. Macintosh v8 or v9

   Since sed is a command-line utility, it is not customary to think
   of sed being used on a Mac. Nonetheless, the following instructions
   from Aurelio Jargas describe the process for running sed on MacOS
   version version 8 or 9.

   (1) Download and install the Apple DiskCopy application

       ftp://ftp.apple.com/developer/Development_Kits

   (2) Download and install Apple MPW

       ftp://ftp.apple.com/developer/Tool_Chest/Core_Mac_OS_Tools/MPW_etc./

   (3) Download and expand Matthias Neeracher's GNU sed for MPW. (They
   seem to have misnumbered the sed filename.)

       ftp://sunsite.cnlab-switch.ch/software/platform/macos/src/mpw_c/sed-2.03.sit.bin

   (4) Enter the sed-3.02 directory and doubleclick the 'sed' file

   (5) MPW Shell will open up. It will be a command window instead of
   a command line, but sed should work as expected. For example:

       echo aa | sed 's/a/Z/g'<ENTER>

   Note that ENTER is different from RETURN on an iMac. Apple *also*
   has its own version of sed on MPW, called "StreamEdit", with a
   syntax fairly similar to that of normal sed.

2.2.2. Shareware and Commercial versions

2.2.2.1. Unix platforms

       [ Additional information needed. ]

2.2.2.2. OS/2

   Hamilton Labs:
       http://www.hamiltonlabs.com/cshell.htm

   A sizable set of Unix/C shell utilities designed for OS/2. Price is
   $350 in the US, $395 elsewhere, with FedEx shipping, unconditional
   guarantee, unlimited support and free updates. A demo version of
   the suite can be downloaded from this site, but a stand-alone copy
   of sed is not available.

2.2.2.3. Windows 95/98, Windows NT, Windows 2000

   Hamilton Labs:
       http://www.hamiltonlabs.com/cshell.htm

   A sizable set of Unix/C shell utilities designed for Win9x, WinNT,
   and Win2K. Price is $350 in the US, $395 elsewhere, with FedEx
   shipping, unconditional guarantee, unlimited support and free
   updates. A demo version of the suite can be downloaded from this
   site, but a stand-alone copy of sed is not available.

   Interix:
       http://www.interix.com

   Interix (formerly known as OpenNT) is advertised as "a complete
   UNIX system environment running natively on Microsoft Windows NT",
   and is licensed and supported by Softway Systems. It offers over
   200 Unix utilities, and supports Unix shells, sockets, networking,
   and more. A single-user edition runs about $200. A free demo or
   evaluation copy will run for 31 days and then quit; to continue
   using it, you must purchase the commercial version.

   MKS NuTCRACKER Professional
       http://www.datafocus.com/products/nutc/

   A different, yet related product line offered by MKS (Mortice Kern
   Systems, below); the awkward spelling "NuTCRACKER" is intentional.
   Various packages offer hundreds of Unix utilities for Win32
   environments. Sed is not available as a separate product.

   UnixDos:
       http://www.unixdos.com

   UnixDos is a suite of 82 Unix utilities ported over to the Windows
   environments. There are 16-bit versions for Win3.x and 32-bit
   versions for WinNT/Win95. It is distributed as uncrippled shareware
   for the first 30 days. After the test period, the utilities will
   not run and you must pay the registration fee of $50.

   Their version of sed supports "\n" in the RHS of expressions, and
   increases the length of input lines to 10,000 characters. By
   special arrangement with the owners, persons who want a licensed
   version of sed *only* (without the other utilities) may pay a
   license fee of $10.

   U/WIN:
       http://www.research.att.com/sw/tools/uwin/

   U/WIN is a suite of Unix utilities created for WinNT and Win95
   systems. It is owned by AT&T, created by David Korn (author of the
   Unix korn shell), and is freely distributed only to educational
   institutions, AT&T employees, or certain researchers; all others
   must pay a fee after a 90-day evaluation period expires. U/WIN
   operates best with the NTFS (WinNT file system) but will run in
   degraded mode with the FAT file system and in further degraded mode
   under Win95. A minimal installation takes about 25 to 30 megs of
   disk space. Sed is not available as a separate file for download,
   but comes with the suite.

2.2.2.4. MS-DOS

   Mix C/Utilities Toolchest
       http://www.mixsoftware.com/product/utility.htm

   According to their web page, "The C/Utilities Toolchest adds over
   40 powerful UNIX utilities to your MS-DOS operating system. The
   result is an environment very similar to UNIX operating systems,
   yet 100% compatible with MS-DOS programs and commands." The
   toolchest costs $19.95, with source code available for an
   additional fee. Mix C's version of sed is not available separately.

   MKS (Mortice Kern Systems) Toolkit
       http://www.mks.com

   Sed comes bundled with the MKS Toolkit, which is distributed only
   as commercial software; it is not available separately.

   Thompson Automation Software
       http://www.tasoft.com

   The Thompson Toolkit contains over 100 familiar Unix utilities,
   including a version of the Unix Korn shell. It runs under MS-DOS,
   OS/2, Win3.x, Win9x, and WinNT. Sed is one of the utilities, though
   Thompson is better known for its version of awk for DOS, TAWK. The
   toolkit runs about $150; sed is not available separately.

2.3. Where can I learn to use sed?

2.3.1. Books

       _Sed & Awk, 2d edition_, by Dale Dougherty & Arnold Robbins
       (Sebastopol, Calif: O'Reilly and Associates, 1997)
       ISBN 1-56592-225-5
       http://www.oreilly.com/catalog/sed2/noframes.html

   About 40 percent of this book is devoted to sed, and maybe 50
   percent is devoted to awk. The other 10 percent covers regexes and
   concepts common to both tools. If you prefer hard copy, this is
   definitely the best single place to learn to use sed, including its
   advanced features.

   The first edition is also very useful. Several typos crept into the
   first printing of the first edition (though if you follow the
   tutorials closely, you'll recognize them right away). A list of
   errors from the first printing of _sed & awk_ is available at
   <http://www.cs.colostate.edu/~dzubera/sedawk.txt>, and errors in
   the 2nd are at <http://www.cs.colostate.edu/~dzubera/sedawk2.txt>,
   though most of these were corrected in later printings. The second
   edition tells how POSIX standards have affected these tools and
   covers the popular GNU versions of sed and awk. Price is about (US)
   $30.00

   -----

       _Mastering Regular Expressions, 2d ed.,_ by Jeffrey E. F. Friedl
       (Sebastopol, Calif: O'Reilly and Associates, 2002)
       ISBN 0-596-00289-0
       http://regex.info
       http://www.oreilly.com/catalog/regex2/
       http://public.yahoo.com/~jfriedl/regex/ (for the first edition)

   Knowing how to use "regular expressions" is essential to effective
   use of most Unix tools. This book focuses on how regular
   expressions can be best implemented in utilities such as perl, vi,
   emacs, and awk, but also touches on sed as well. Friedl's home page
   (above) gives links to other sites which help students learn to
   master regular expressions. His site also gives a Perl script for
   determining a syntactically valid e-mail address, using regexes:

       http://public.yahoo.com/~jfriedl/regex/code.html

   -----

       _Awk und Sed_, by Helmut Herold.
       (Bonn: Addison-Wesley, 1994; 288 pages)
       2nd edition to be released in March 2003
       ISBN 3-8273-2094-1
       http://www.addison-wesley.de/main/main.asp?page=home/bookdetails&ProductID=37214

2.3.2. Mailing list

   If you are interested in learning more about sed (its syntax, using
   regular expressions, etc.) you are welcome to subscribe to a
   sed-oriented mailing list. In fact, there are two mailing lists
   about sed: one in English named "sed-users", moderated by Sven
   Guckes; and one in Portuguese named "sed-BR" (for sed-Brazil),
   moderated by Aurelio Marinho Jargas. The average volume of mail for
   "sed-users" is about 35 messages a week; the average volume of mail
   for "sed-BR" is about 15 messages a week.

       sed-BR mailing list:    http://br.groups.yahoo.com/group/sed-br/
       sed-users mailing list: http://groups.yahoo.com/group/sed-users/

   To subscribe to sed-users, send a blank message to:

       sed-users-subscribe@yahoogroups.com

   To unsubscribe from sed-users, send a blank message to:

       sed-users-unsubscribe@yahoogroups.com

2.3.3. Tutorials, electronic text

   The original users manual for sed, by Lee E. McMahon, from the
   7th edition UNIX Manual (1978), with the classic "Kubla Khan"
   example and tutorial, in formatted text format:
       http://sed.sourceforge.net/grabbag/tutorials/sed_mcmahon.txt

   The source code to the preceding manual. Use "troff -ms sed" to
   print this file properly:
       http://plan9.bell-labs.com/7thEdMan/vol2/sed
       http://cm.bell-labs.com/7thEdMan/vol2/sed

   "Do It With Sed", by Carlos Duarte
       http://www.dbnet.ece.ntua.gr/~george/sed/OLD/sedtut_1.html

   "Sed: How to use sed, a special editor for modifying files
   automatically", by Bruce Barnett and General Electric Company
       http://www.grymoire.com/Unix/Sed.html

   U-SEDIT2.ZIP, by Mike Arst (16 June 1990)
       ftp://ftp.cs.umu.se/pub/pc/u-sedit2.zip
       ftp://ftp.uni-stuttgart.de/pub/systems/msdos/util/unixlike/u-sedit2.zip
       ftp://sunsite.icm.edu.pl/vol/wojsyl/garbo/pc/editor/u-sedit2.zip
       ftp://ftp.sogang.ac.kr/pub/msdos/garbo_pc/editor/u-sedit2.zip

   U-SEDIT3.ZIP, by Mike Arst (24 Jan. 1992)
       http://www.student.northpark.edu/pemente/sed/u-sedit3.zip
       CompuServe DTPFORUM, "PC DTP Utilities" library, file SEDDOC.ZIP

   Another sed FAQ
       http://www.dreamwvr.com/sed-info/sed-faq.html

   sed-tutorial, by Felix von Leitner
       http://www.math.fu-berlin.de/~leitner/sed/tutorial.html

   "Manipulating text with sed," chapter 14 of the SCO OpenServer
   "Operating System Users Guide"
       http://ou800doc.caldera.com/SHL_automate/CTOC-Manipulating_text_with_sed.html

   "Combining the Bourne-shell, sed and awk in the UNIX environment
   for language analysis," by Lothar Schmitt and Kiel Christianson.
   This basic tutorial on the Bourne shell, sed and awk downloads as a
   71-page PostScript file (compressed to 290K with gzip). You may
   need to navigate down from the root to get the file.
       ftp://ftp.u-aizu.ac.jp/u-aizu/doc/Tech-Report/1997/97-2-007.tar.gz
       available upon request from Lothar Schmitt <lothar@u-aizu.ac.jp>

2.3.4. General web and ftp sites

       http://sed.sourceforge.net/grabbag             # Collected scripts
       http://main.rtfiber.com.tw/~changyj/sed/       # Yao-Jen Chang
       http://www.math.fu-berlin.de/~guckes/sed/      # Sven Guckes
       http://www.math.fu-berlin.de/~leitner/sed/     # Felix von Leitner
       http://www.dbnet.ece.ntua.gr/~george/sed/      # Yiorgos Adamopoulos
       http://www.student.northpark.edu/pemente/sed/  # Eric Pement

       http://spacsun.rice.edu/FAQ/sed.html
       ftp://algos.inesc.pt/pub/users/cdua/scripts.tar.gz (sed and shell scripts)

   "Handy One-Liners For Sed", compiled by Eric Pement. A large list
   of 1-line sed commands which can be executed from the command line.
       http://sed.sourceforge.net/sed1line.txt
       http://www.student.northpark.edu/pemente/sed/sed1line.txt

   "Handy One-Liners For Sed", translated to Portuguese
       http://wmaker.lrv.ufsc.br/sed_ptBR.html

   The Single UNIX Specification, Version 3 (technical man page)
       http://www.opengroup.org/onlinepubs/007904975/utilities/sed.html

   Getting started with sed
       http://www.cs.hmc.edu/tech_docs/qref/sed.html

   masm to gas converter
       http://www.delorie.com/djgpp/faq/converting/asm2s-sed.html

   mail2html.zip
       http://www.crispen.org/src/#mail2html

   sample uses of sed in batch files and scripts (Benny Pederson)
       http://users.cybercity.dk/~bse26236/batutil/help/SED.HTM

   dc.sed - the most complex and impressive sed script ever written.
   This sed script by Greg Ubben emulates the Unix dc (desk
   calculator), including base conversion, exponentiation, square
   roots, and much more.
       http://sed.sourceforge.net/grabbag/scripts/dc_overview.htm

   If you should find other tutorials or scripts that should be added
   to this document, please forward the URLs to the FAQ maintainer.

------------------------------

3. TECHNICAL

3.1. More detailed explanation of basic sed

   Sed takes a script of editing commands and applies each command, in
   order, to each line of input. After all the commands have been
   applied to the first line of input, that line is output. A second
   input line is taken for processing, and the cycle repeats. Sed
   scripts can address a single line by line number or by matching a
   /RE pattern/ on the line. An exclamation mark '!' after a regex
   ('/RE/!') or line number will select all lines that do NOT match
   that address. Sed can also address a range of lines in the same
   manner, using a comma to separate the 2 addresses.

     $d               # delete the last line of the file
     /[0-9]\{3\}/p    # print lines with 3 consecutive digits
     5!s/ham/cheese/  # except on line 5, replace 'ham' with 'cheese'
     /awk/!s/aaa/bb/  # unless 'awk' is found, replace 'aaa' with 'bb'
     17,/foo/d        # delete all lines from line 17 up to 'foo'

   Following an address or address range, sed accepts curly braces
   '{...}' so several commands may be applied to that line or to the
   lines matched by the address range. On the command line, semicolons
   ';' separate each instruction and must precede the closing brace.

     sed '/Owner:/{s/yours/mine/g;s/your/my/g;s/you/me/g;}' file

   Range addresses operate differently depending on which version of
   sed is used (see section 3.4, below). For further information on
   using sed, consult the references in section 2.3, above.

3.1.1. Regular expressions on the left side of "s///"

   All versions of sed support Basic Regular Expressions (BREs). For
   the syntax of BREs, enter "man ed" at a Unix shell prompt. A
   technical description of BREs from IEEE POSIX 1003.1-2001 and the
   Single UNIX Specification Version 3 is available online at:
   http://www.opengroup.org/onlinepubs/007904975/basedefs/xbd_chap09.html#tag_09_03

   Sed normally supports BREs plus '\n' to match a newline in the
   pattern space, plus '\xREx' as equivalent to '/RE/', where 'x' is any
   character other than a newline or another backslash.

   Some versions of sed support supersets of BREs, or "extended
   regular expressions", which offer additional metacharacters for
   increased flexibility. For additional information on extended REs
   in GNU sed, see sections 3.7 ("GNU/POSIX extensions to regular
   expressions") and 6.7.3 ("Special syntax in REs"), below.

   Though not required by BREs, some versions of sed support \t to
   represent a TAB, \r for carriage return, \xHH for direct entry of
   hex codes, and so forth. Other versions of sed do not.

   ssed (super-sed) introduced many new features for LHS pattern
   matching, too many to give here. The complete list is found in
   section 6.7.3.H ("ssed"), below.

3.1.2. Escape characters on the right side of "s///"

   The right-hand side (the replacement part) in "s/find/replace/" is
   almost always a string literal, with no interpolation of these
   metacharacters:

       .   ^   $   [   ]   {   }   (   )  ?   +   *   |

   Three things *are* interpolated: ampersand (&), backreferences, and
   options for special seds. An ampersand on the RHS is replaced by
   the entire expression matched on the LHS. There is _never_ any
   reason to use grouping like this:

       s/\(some-complex-regex\)/one two \1 three/

   since you can do this instead:

       s/some-complex-regex/one two & three/

   To enter a literal ampersand on the RHS, type '\&'.

   Grouping and backreferences: All versions of sed support grouping
   and backreferences on the LHS and backreferences only on the RHS.
   Grouping allows a series of characters to be collected in a set,
   indicating the boundaries of the set with \( and \). Then the set
   can be designated to be repeated a certain number of times

       \(like this\)*   or   \(like this\)\{5,7\}.

   Groups can also be nested "\(like \(this\) is here\)" and may
   contain any valid RE. Backreferences repeat the contents of a
   particular group, using a backslash and a digit (1-9) for each
   corresponding group. In other words, "/\(pom\)\1/" is another way
   of writing "/pompom/". If groups are nested, backreference numbers
   are counted by matching \( in strict left to right order.  Thus,
   /..\(the \(word\)\) \("foo"\)../ is matched by the backreference
   \3. Backreferences can be used in the LHS, the RHS, and in normal
   RE addressing (see section 3.3).  Thus,

       /\(.\)\1\(.\)\2\(.\)\3/;      # matches "bookkeeper"
       /^\(.\)\(.\)\(.\)\3\2\1$/;    # finds 6-letter palindromes

   Seds differ in how they treat invalid backreferences where no
   corresponding group occurs. To insert a literal ampersand or
   backslash into the RHS, prefix it with a backslash: \& or \\.

   ssed, sed16, and sedmod permit additional options on the RHS. They
   all support changing part of the replacement string to upper case
   (\u or \U), lower case (\l or \L), or to end case conversion (\E).
   Both sed16 and sedmod support awk-style word references ($1, $2,
   $3, ...) and $0 to insert the entire line before conversion.

     echo ab ghi | sed16 "s/.*/$0 - \U$2/"   # prints "ab ghi - GHI"

   *Note:* This feature of sed16 and sedmod will break sed scripts which
   put a dollar sign and digit into the RHS. Though this is an unlikely
   combination, it's worth remembering if you use other people's scripts.

3.1.3.  Substitution switches

   Standard versions of sed support 4 main flags or switches which may
   be added to the end of an "s///" command. They are:

       N      - Replace the Nth match of the pattern on the LHS, where
                N is an integer between 1 and 512. If N is omitted,
                the default is to replace the first match only.
       g      - Global replace of all matches to the pattern.
       p      - Print the results to stdout, even if -n switch is used.
       w file - Write the pattern space to 'file' if a replacement was
                done. If the file already exists when the script is
                executed, it is overwritten. During script execution,
                w appends to the file for each match.

   GNU sed 3.02 and ssed also offer the /I switch for doing a
   case-insensitive match. For example,

     echo ONE TWO | gsed "s/one/unos/I"      # prints "unos TWO"

   GNU sed 4.x and ssed add the /M switch, to simplify working with
   multi-line patterns: when it is used, ^ or $ will match BOL or EOL.
   \` and \' remain available to match the start and end of pattern
   space, respectively.

   ssed supports two more switches, /S and /X, when its Perl mode is
   used. They are described in detail in section 6.7.3.H, below.

3.1.4. Command-line switches

   All versions of sed support two switches, -e and -n. Though sed
   usually separates multiple commands with semicolons (e.g., "H;d;"),
   certain commands could not accept a semicolon command separator.
   These include :labels, 't', and 'b'. These commands had to occur
   last in a script, separated by -e option switches. For example:

     # The 'ta' means jump to label :a if last s/// returns true
     sed -e :a -e '$!N;s/\n=/ /;ta' -e 'P;D' file

   The -n switch turns off sed's default behavior of printing every
   line. With -n, lines are printed only if explicitly told to. In
   addition, for certain versions of sed, if an external script begins
   with "#n" as its first two characters, the output is suppressed
   (exactly as if -n had been entered on the command line). A list of
   which versions appears in section 6.7.2., below.

   GNU sed 4.x and ssed support additional switches. -l (lowercase L),
   followed by a number, lets you adjust the default length of the 'l'
   and 'L' commands (note that these implementations of sed also
   support an argument to these commands, to tailor the length
   separately of each occurrence of the command).

   -i activates in-place editing (see section 4.41.1, below). -s
   treats each file as a separate stream: sed by default joins all the
   files, so $ represents the last line of the last file; 15 means the
   15th line in the joined stream; and /abc/,/def/ might match across
   files.

   When -s is used, however all addresses refer to single files. For
   example, $ represents the last line of each input file; 15 means
   the 15th line of each input file; and /abc/,/def/ will be "reset"
   (in other words, sed will not execute the commands and start
   looking for /abc/ again) if a file ends before /def/ has been
   matched. Note that -i automatically activates this interpretation
   of addresses.

3.2. Common one-line sed scripts

   A separate document of over 70 handy "one-line" sed commands is
   available at
       http://sed.sourceforge.net/sed1line.txt

   Here are several common sed commands for one-line use. MS-DOS users
   should replace single quotes ('...') with double quotes ("...") in
   these examples. A specific filename usually follows the script,
   though the input may also come via piping or redirection.

   # Double space a file
   sed G file

   # Triple space a file
   sed 'G;G' file

   # Under UNIX: convert DOS newlines (CR/LF) to Unix format
   sed 's/.$//' file    # assumes that all lines end with CR/LF
   sed 's/^M$// file    # in bash/tcsh, press Ctrl-V then Ctrl-M

   # Under DOS: convert Unix newlines (LF) to DOS format
   sed 's/$//' file                     # method 1
   sed -n p file                        # method 2

   # Delete leading whitespace (spaces/tabs) from front of each line
   # (this aligns all text flush left). '^t' represents a true tab
   # character. Under bash or tcsh, press Ctrl-V then Ctrl-I.
   sed 's/^[ ^t]*//' file

   # Delete trailing whitespace (spaces/tabs) from end of each line
   sed 's/[ ^t]*$//' file               # see note on '^t', above

   # Delete BOTH leading and trailing whitespace from each line
   sed 's/^[ ^t]*//;s/[ ^]*$//' file    # see note on '^t', above

   # Substitute "foo" with "bar" on each line
   sed 's/foo/bar/' file        # replaces only 1st instance in a line
   sed 's/foo/bar/4' file       # replaces only 4th instance in a line
   sed 's/foo/bar/g' file       # replaces ALL instances within a line

   # Substitute "foo" with "bar" ONLY for lines which contain "baz"
   sed '/baz/s/foo/bar/g' file

   # Delete all CONSECUTIVE blank lines from file except the first.
   # This method also deletes all blank lines from top and end of file.
   # (emulates "cat -s")
   sed '/./,/^$/!d' file       # this allows 0 blanks at top, 1 at EOF
   sed '/^$/N;/\n$/D' file     # this allows 1 blank at top, 0 at EOF

   # Delete all leading blank lines at top of file (only).
   sed '/./,$!d' file

   # Delete all trailing blank lines at end of file (only).
   sed -e :a -e '/^\n*$/{$d;N;};/\n$/ba' file

   # If a line ends with a backslash, join the next line to it.
   sed -e :a -e '/\\$/N; s/\\\n//; ta' file

   # If a line begins with an equal sign, append it to the previous
   # line (and replace the "=" with a single space).
   sed -e :a -e '$!N;s/\n=/ /;ta' -e 'P;D' file

3.3. Addressing and address ranges

   Sed commands may have an optional "address" or "address range"
   prefix. If there is no address or address range given, then the
   command is applied to all the lines of the input file or text
   stream. Three commands cannot take an address prefix:

      - labels, used to branch or jump within the script
      - the close brace, '}', which ends the '{' "command"
      - the '#' comment character, also technically a "command"

   An address can be a line number (such as 1, 5, 37, etc.), a regular
   expression (written in the form /RE/ or \xREx where 'x' is any
   character other than '\' and RE is the regular expression), or the
   dollar sign ($), representing the last line of the file. An
   exclamation mark (!) after an address or address range will apply
   the command to every line EXCEPT the ones named by the address. A
   null regex ("//") will be replaced by the last regex which was
   used. Also, some seds do not support \xREx as regex delimiters.

     5d               # delete line 5 only
     5!d              # delete every line except line 5
     /RE/s/LHS/RHS/g  # substitute only if RE occurs on the line
     /^$/b label      # if the line is blank, branch to ':label'
     /./!b label      # ... another way to write the same command
     \%.%!b label     # ... yet another way to write this command
     $!N              # on all lines but the last, get the Next line

   Note that an embedded newline can be represented in an address by
   the symbol \n, but this syntax is needed only if the script puts 2
   or more lines into the pattern space via the N, G, or other
   commands. The \n symbol does *not* match the newline at an
   end-of-line because when sed reads each line into the pattern space
   for processing, it strips off the trailing newline, processes the
   line, and adds a newline back when printing the line to standard
   output. To match the end-of-line, use the '$' metacharacter, as
   follows:

     /tape$/       # matches the word 'tape' at the end of a line
     /tape$deck/   # matches the word 'tape$deck' with a literal '$'
     /tape\ndeck/  # matches 'tape' and 'deck' with a newline between

   The following sed commands usually accept *only* a single address.
   All other commands (except labels, '}', and '#') accept both single
   addresses and address ranges.

     =       print to stdout the line number of the current line
     a       after printing the current line, append "text" to stdout
     i       before printing the current line, insert "text" to stdout
     q       quit after the current line is matched
     r file  prints contents of "file" to stdout after line is matched

   Note that we said "usually." If you need to apply the '=', 'a',
   'i', or 'r' commands to each and every line within an address
   range, this behavior can be coerced by the use of braces. Thus,
   "1,9=" is an invalid command, but "1,9{=;}" will print each line
   number followed by its line for the first 9 lines (and then print
   the rest of the rest of the file normally).

   Address ranges occur in the form

       <address1>,<address2>    or    <address1>,<address2>!

   where the address can be a line number or a standard /regex/.
   <address2> can also be a dollar sign, indicating the end of file.
   Under GNU sed 3.02+, ssed, and sed15+, <address2> may also be a
   notation of the form +num, indicating the next _num_ lines after
   <address1> is matched.

   Address ranges are:

   (1) Inclusive. The range "/From here/,/eternity/" matches all the
   lines containing "From here" up to and including the line
   containing "eternity". It will not stop on the line just prior to
   "eternity". (If you don't like this, see section 4.24.)

   (2) Plenary. They always match full lines, not just parts of lines.
   In other words, a command to change or delete an address range will
   change or delete whole lines; it won't stop in the middle of a
   line.

   (3) Multi-linear. Address ranges normally match 2 lines or more.
   The second address will never match the same line the first address
   did; therefore a valid address range always spans at least two
   lines, with these exceptions which match only one line:

      - if the first address matches the last line of the file
      - if using the syntax "/RE/,3" and /RE/ occurs only once in the
        file at line 3 or below
      - if using HHsed v1.5. See section 3.4.

   (4) Minimalist. In address ranges with /regex/ as <address2>, the
   range "/foo/,/bar/" will stop at the first "bar" it finds, provided
   that "bar" occurs on a line below "foo". If the word "bar" occurs
   on several lines below the word "foo", the range will match all the
   lines from the first "foo" up to the first "bar". It will not
   continue hopping ahead to find more "bar"s. In other words, address
   ranges are not "greedy," like regular expressions.

   (5) Repeating. An address range will try to match more than one
   block of lines in a file. However, the blocks cannot nest. In
   addition, a second match will not "take" the last line of the
   previous block.  For example, given the following text,

       start
       stop  start
       stop

   the sed command '/start/,/stop/d' will only delete the first two
   lines. It will not delete all 3 lines.

   (6) Relentless. If the address range finds a "start" match but
   doesn't find a "stop", it will match every line from "start" to the
   end of the file. Thus, beware of the following behaviors:

     /RE1/,/RE2/  # If /RE2/ is not found, matches from /RE1/ to the
                  # end-of-file.

     20,/RE/      # If /RE/ is not found, matches from line 20 to the
                  # end-of-file.

     /RE/,30      # If /RE/ occurs any time after line 30, each
                  # occurrence will be matched in sed15+, sedmod, and
                  # GNU sed v3.02+. GNU sed v2.05 and 1.18 will match
                  # from the 2nd occurrence of /RE/ to the end-of-file.

   If these behaviors seem strange, remember that they occur because
   sed does not look "ahead" in the file. Doing so would stop sed from
   being a stream editor and have adverse effects on its efficiency.
   If these behaviors are undesirable, they can be circumvented or
   corrected by the use of nested testing within braces. The following
   scripts work under GNU sed 3.02:

     # Execute your_commands on range "/RE1/,/RE2/", but if /RE2/ is
     # not found, do nothing.
     /RE1/{:a;N;/RE2/!ba;your_commands;}

     # Execute your_commands on range "20,/RE/", but if /RE/ is not
     # found, do nothing.
     20{:a;N;/RE/!ba;your_commands;}

   As a side note, once we've used N to "slurp" lines together to test
   for the ending expression, the pattern space will have gathered
   many lines (possibly thousands) together and concatenated them as a
   single expression, with the \n sequence marking line breaks. The
   REs *within* the pattern space may have to be modified (e.g., you
   must write '/\nStart/' instead of '/^Start/' and '/[^\n]*/' instead
   of '/.*/') and other standard sed commands will be unavailable or
   difficult to use.

     # Execute your_commands on range "/RE/,30", but if /RE/ occurs
     # on line 31 or later, do not match it.
     1,30{/RE/,$ your_commands;}

   For related suggestions on using address ranges, see sections 4.2,
   4.15, and 4.19 of this FAQ. Also, note the following section.

3.4. Address ranges in GNU sed and HHsed

   (1) GNU sed 3.02+, ssed, and sed15+ all support address ranges like:

       /regex/,+5

   which match /regex/ plus the next 5 lines (or EOF, whichever comes
   first).

   (2) GNU sed v3.02.80 (and above) and ssed support address ranges of:

       0,/regex/

   as a special case to permit matching /regex/ if it occurs on the
   first line. This syntax permits a range expression that matches
   every line from the top of the file to the first instance of
   /regex/, even if /regex/ is on the first line.

   (3) HHsed (sed15) has an exceptional way of implementing

       /regex1/,/regex2/

   If /RE1/ and /RE2/ both occur on the *same* line, HHsed will match
   that single line. In other words, an address range block can
   consist of just one line. HHsed will then look for the next
   occurrence of /regex1/ to begin the block again.

   Every other version of sed (including sed16) requires 2 lines to
   match an address range, and thus /regex1/ and /regex2/ cannot
   successfully match just one line. See also the comments at
   section 7.9.4, below.

   (4) BEGIN~STEP selection: ssed and GNU sed (v2.05 and above) offer
   a form of addressing called "BEGIN~STEP selection". This is *not* a
   range address, which selects an inclusive block of consecutive
   lines from /start/ to /finish/. But I think it seems to belong here.

   Given an expression of the form "M~N", where M and N are integers,
   GNU sed and ssed will select every Nth line, beginning at line M.
   (With gsed v2.05, M had to be less than N, but this restriction is
   no longer necessary). Both M and N may equal 0 ("0~0" selects every
   line). These examples illustrate the syntax:

     sed '1~3d' file      # delete every 3d line, starting with line 1
                          # deletes lines 1, 4, 7, 10, 13, 16, ...

     sed '0~3d' file      # deletes lines 3, 6, 9, 12, 15, 18, ...

     sed -n '2~5p' file   # print every 5th line, starting with line 2
                          # prints lines 2, 7, 12, 17, 22, 27, ...

   (5) Finally, GNU sed v2.05 has a bug in range addressing (see
   section 7.5), which was fixed in the higher versions.


3.5. Debugging sed scripts

   The following two debuggers should make it easier to understand how
   sed scripts operate. They can save hours of grief when trying to
   determine the problems with a sed script.

   (1) sd (sed debugger), by Brian Hiles

   This debugger runs under a Unix shell, is powerful, and is easy to
   use. sd has conditional breakpoints and spypoints of the pattern
   space and hold space, on any scope defined by regex match and/or
   script line number. It can be semi-automated, can save diagnostic
   reports, and shows potential problems with a sed script before it
   tries to execute it. The script is robust and requires the Unix
   shell utilities plus the Bourne shell or Korn shell to execute.

       http://sed.sourceforge.net/grabbag/scripts/sd.ksh.txt (2003)
       http://sed.sourceforge.net/grabbag/scripts/sd.sh.txt  (1998)

   (2) sedsed, by Aurelio Jargas

   This debugger requires Python to run it, and it uses your own
   version of sed, whatever that may be. It displays the current input
   line, the pattern space, and the hold space, before and after each
   sed command is executed.

       http://sedsed.sourceforge.net


3.6. Notes about s2p, the sed-to-perl translator

   s2p (sed to perl) is a Perl program to convert sed scripts into the
   Perl programming language; it is included with many versions of
   Perl. These problems have been found when using s2p:

   (1) Doesn't recognize the semicolon properly after s/// commands.

       s/foo/bar/g;

   (2) Doesn't trim trailing whitespace after s/// commands. Even lone
   trailing spaces, without comments, produce an error.

   (3) Doesn't handle multiple commands within braces. E.g.,

       1,4{=;G;}

   will produce perl code with missing braces, and miss the second "G"
   command as well. In fact, any commands after the first one are
   missed in the perl output script, and the output perl script will
   also contain mismatched braces.

3.7. GNU/POSIX extensions to regular expressions

   GNU sed supports "character classes" in addition to regular
   character sets, such as [0-9A-F]. Like regular character sets,
   character classes represent any single character within a set.

   "Character classes are a new feature introduced in the POSIX
   standard. A character class is a special notation for describing
   lists of characters that have a specific attribute, but where the
   actual characters themselves can vary from country to country
   and/or from character set to character set. For example, the notion
   of what is an alphabetic character differs in the USA and in
   France." [quoted from the docs for GNU awk v3.1.0.]

   Though character classes don't generally conserve space on the
   line, they help make scripts portable for international use. The
   equivalent character sets _for U.S. users_ follows:

     [[:alnum:]]  - [A-Za-z0-9]     Alphanumeric characters
     [[:alpha:]]  - [A-Za-z]        Alphabetic characters
     [[:blank:]]  - [ \x09]         Space or tab characters only
     [[:cntrl:]]  - [\x00-\x19\x7F] Control characters
     [[:digit:]]  - [0-9]           Numeric characters
     [[:graph:]]  - [!-~]           Printable and visible characters
     [[:lower:]]  - [a-z]           Lower-case alphabetic characters
     [[:print:]]  - [ -~]           Printable (non-Control) characters
     [[:punct:]]  - [!-/:-@[-`{-~]  Punctuation characters
     [[:space:]]  - [ \t\v\f]       All whitespace chars
     [[:upper:]]  - [A-Z]           Upper-case alphabetic characters
     [[:xdigit:]] - [0-9a-fA-F]     Hexadecimal digit characters

   Note that [[:graph:]] does not match the space " ", but [[:print:]]
   does. Some character classes may (or may not) match characters in
   the high ASCII range (ASCII 128-255 or 0x80-0xFF), depending on
   which C library was used to compile sed. For non-English languages,
   [[:alpha:]] and other classes may also match high ASCII characters.

------------------------------

4. EXAMPLES

   ONE-CHARACTER QUESTIONS

4.1. How do I insert a newline into the RHS of a substitution?

   Several versions of sed permit '\n' to be typed directly into the
   RHS, which is then converted to a newline on output: ssed,
   gsed302a+, gsed103 (with the -x switch), sed15+, sedmod, and
   UnixDOS sed. The _easiest_ solution is to use one of these
   versions.

   For other versions of sed, try one of the following:

   (a) If typing the sed script from a Bourne shell, use one backslash
   "\" if the script uses 'single quotes' or two backslashes "\\" if
   the script requires "double quotes". In the example below, note
   that the leading '>' on the 2nd line is generated by the shell to
   prompt the user for more input. The user types in slash,
   single-quote, and then ENTER to terminate the command:

     [sh-prompt]$ echo twolines | sed 's/two/& new\
     >/'
     two new
     lines
     [bash-prompt]$

   (b) Use a script file with one backslash '\' in the script,
   immediately followed by a newline. This will embed a newline into
   the "replace" portion. Example:

     sed -f newline.sed files

     # newline.sed
     s/twolines/two new\
     lines/g

   Some versions of sed may not need the trailing backslash. If so,
   remove it.

   (c) Insert an unused character and pipe the output through tr:

     echo twolines | sed 's/two/& new=/' | tr "=" "\n"   # produces
     two new
     lines

   (d) Use the "G" command:

   G appends a newline, plus the contents of the hold space to the end
   of the pattern space. If the hold space is empty, a newline is
   appended anyway. The newline is stored in the pattern space as "\n"
   where it can be addressed by grouping "\(...\)" and moved in the
   RHS. Thus, to change the "twolines" example used earlier, the
   following script will work:

     sed '/twolines/{G;s/\(two\)\(lines\)\(\n\)/\1\3\2/;}'

   (e) Inserting full lines, not breaking lines up:

   If one is not *changing* lines but only inserting complete lines
   before or after a pattern, the procedure is much easier. Use the
   "i" (insert) or "a" (append) command, making the alterations by an
   external script. To insert "This line is new" BEFORE each line
   matching a regex:

     /RE/i This line is new               # HHsed, sedmod, gsed 3.02a
     /RE/{x;s/$/This line is new/;G;}     # other seds

   The two examples above are intended as "one-line" commands entered
   from the console. If using a sed script, "i\" immediately followed
   by a literal newline will work on all versions of sed. Furthermore,
   the command "s/$/This line is new/" will only work if the hold
   space is already empty (which it is by default).

   To append "This line is new" AFTER each line matching a regex:

     /RE/a This line is new               # HHsed, sedmod, gsed 3.02a
     /RE/{G;s/$/This line is new/;}       # other seds

   To append 2 blank lines after each line matching a regex:

     /RE/{G;G;}                    # assumes the hold space is empty

   To replace each line matching a regex with 5 blank lines:

     /RE/{s/.*//;G;G;G;G;}         # assumes the hold space is empty

   (f) Use the "y///" command if possible:

   On some Unix versions of sed (not GNU sed!), though the s///
   command won't accept '\n' in the RHS, the y/// command does. If
   your Unix sed supports it, a newline after "aaa" can be inserted
   this way (which is not portable to GNU sed or other seds):

     s/aaa/&~/; y/~/\n/;    # assuming no other '~' is on the line!

4.2. How do I represent control-codes or nonprintable characters?

   Several versions of sed support the notation \xHH, where "HH" are
   two hex digits, 00-FF: ssed, GNU sed v3.02.80 and above, GNU sed
   v1.03, sed16 and sed15 (HHsed). Try to use one of those versions.

   Sed is not intended to process binary or object code, and files
   which contain nulls (0x00) will usually generate errors in most
   versions of sed. The latest versions of GNU sed and ssed are an
   exception; they permit nulls in the input files and also in
   regexes.

   On Unix platforms, the 'echo' command may allow insertion of octal
   or hex values, e.g., `echo "\0nnn"` or `echo -n "\0nnn"`. The echo
   command may also support syntax like '\\b' or '\\t' for backspace
   or tab characters. Check the man pages to see what syntax your
   version of echo supports. Some versions support the following:

     # replace 0x1A (32 octal) with ASCII letters
     sed 's/'`echo "\032"`'/Ctrl-Z/g'

     # note the 3 backslashes in the command below
     sed "s/.`echo \\\b`//g"

4.3. How do I convert files with toggle characters, like +this+, to
look like [i]this[/i]?

   Input files, especially message-oriented text files, often contain
   toggle characters for emphasis, like ~this~, *this*, or =this=. Sed
   can make the same input pattern produce alternating output each
   time it is encountered. Typical needs might be to generate HMTL
   codes or print codes for boldface, italic, or underscore. This
   script accomodates multiple occurrences of the toggle pattern on
   the same line, as well as cases where the pattern starts on one
   line and finishes several lines later, even at the end of the file:

     # sed script to convert +this+ to [i]this[/i]
     :a
     /+/{ x;        # If "+" is found, switch hold and pattern space
       /^ON/{       # If "ON" is in the (former) hold space, then ..
         s///;      # .. delete it
         x;         # .. switch hold space and pattern space back
         s|+|[/i]|; # .. turn the next "+" into "[/i]"
         ba;        # .. jump back to label :a and start over
       }
     s/^/ON/;       # Else, "ON" was not in the hold space; create it
     x;             # Switch hold space and pattern space
     s|+|[i]|;      # Turn the first "+" into "[i]"
     ba;            # Branch to label :a to find another pattern
     }
     #---end of script---

   This script uses the hold space to create a "flag" to indicate
   whether the toggle is ON or not. We have added remarks to
   illustrate the script logic, but in most versions of sed remarks
   are not permitted after 'b'ranch commands or labels.

   If you are sure that the +toggle+ characters never cross line
   boundaries (i.e., never begin on one line and end on another), this
   script can be reduced to one line:

     s|+\([^+][^+]*\)+|[i]\1[/i]|g

   If your toggle pattern contains regex metacharacters (such as '*'
   or perhaps '+' or '?'), remember to quote them with backslashes.

   CHANGING STRINGS

4.10. How do I perform a case-insensitive search?

   Several versions of sed support case-insensitive matching: ssed and
   GNU sed v3.02+ (with I flag after s/// or /regex/); sedmod with the
   -i switch; and sed16 (which supports both types of switches).

   With other versions of sed, case-insensitive searching is awkward,
   so people may use awk or perl instead, since these programs have
   options for case-insensitive searches. In gawk/mawk, use "BEGIN
   {IGNORECASE=1}" and in perl, "/regex/i". For other seds, here are
   three solutions:

   Solution 1: convert everything to upper case and search normally

     # sed script, solution 1
     h;          # copy the original line to the hold space
                 # convert the pattern space to solid caps
     y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/
                 # now we can search for the word "CARLOS"
     /CARLOS/ {
          # add or insert lines. Note: "s/.../.../" will not work
          # here because we are searching a modified pattern
          # space and are not printing the pattern space.
     }
     x;          # get back the original pattern space
                 # the original pattern space will be printed
     #---end of sed script---

   Solution 2: search for both cases

   Often, proper names will either start with all lower-case ("unix"),
   with an initial capital letter ("Unix") or occur in solid caps
   ("UNIX"). There may be no need to search for every possibility.

     /UNIX/b match
     /[Uu]nix/b match

   Solution 3: search for all possible cases

     # If you must, search for any possible combination
     /[Ca][Aa][Rr][Ll][Oo][Ss]/ { ... }

   Bear in mind that as the pattern length increases, this solution
   becomes an order of magnitude slower than the one of Solution 1, at
   least with some implementations of sed.

4.11. How do I match only the first occurrence of a pattern?

   (1) The general solution is to use GNU sed or ssed, with one of
   these range expressions. The first script ("print only the first
   match") works with any version of sed:

     sed -n '/RE/{p;q;}' file       # print only the first match
     sed '0,/RE/{//d;}' file        # delete only the first match
     sed '0,/RE/s//to_that/' file   # change only the first match

   (2) If you cannot use GNU sed and if you *know* the pattern will
   not occur on the first line, this will work:

     sed '1,/RE/{//d;}' file        # delete only the first match
     sed '1,/RE/s//to_that/' file   # change only the first match

   (3) If you cannot use GNU sed and the pattern *might* occur on the
   first line, use one of the following commands (credit for short GNU
   script goes to Donald Bruce Stewart):

     sed '/RE/{x;/Y/!{s/^/Y/;h;d;};x;}' file       # delete (one way)
     sed -e '/RE/{d;:a' -e '$!N;$ba' -e '}' file   # delete (another way)
     sed '/RE/{d;:a;N;$ba;}' file                  # same script, GNU sed
     sed -e '/RE/{s//to_that/;:a' -e '$!N;$!ba' -e '}' file  # change

   Still another solution, using a flag in the hold space. This is
   portable to all seds and works if the pattern is on the first line:

     # sed script to change "foo" to "bar" only on the first occurrence
     1{x;s/^/first/;x;}
     1,/foo/{x;/first/s///;x;s/foo/bar/;}
     #---end of script---

4.12. How do I parse a comma-delimited (CSV) data file?

   Comma-delimited data files can come in several forms, requiring
   increasing levels of complexity in parsing and handling. They are
   often referred to as CSV files (for "comma separated values") and
   occasionally as SDF files (for "standard data format"). Note that
   some vendors use "SDF" to refer to variable-length records with
   comma-separated fields which are "double-quoted" if they contain
   character values, while other vendors use "SDF" to designate
   fixed-length records with fixed-length, nonquoted fields! (For help
   with fixed-length fields, see question 4.23)

   The term "CSV" became a de-facto standard when Microsoft Excel used
   it as an optional output file format.

   Here are 4 different forms you may encounter in comma-delimited data:

   (a) No quotes, no internal commas

       1001,John Smith,PO Box 123,Chicago,IL,60699
       1002,Mary Jones,320 Main,Denver,CO,84100,

   (b) Like (a), with quotes around each field

       "1003","John Smith","PO Box 123","Chicago","IL","60699"
       "1004","Mary Jones","320 Main","Denver","CO","84100"

   (c) Like (b), with embedded commas

       "1005","Tom Hall, Jr.","61 Ash Ct.","Niles","OH","44446"
       "1006","Bob Davis","429 Pine, Apt. 5","Boston","MA","02128"

   (d) Like (c), with embedded commas and quotes

       "1007","Sue "Red" Smith","19 Main","Troy","MI","48055"
       "1008","Joe "Hey, guy!" Hall","POB 44","Reno","NV","89504"

   In each example above, we have 7 fields and 6 commas which function
   as field separators. Case (c) is a very typical form of these data
   files, with double quotes used to enclose each field and to protect
   internal commas (such as "Tom Hall, Jr.") from interpretation as
   field separators. However, many times the data may include both
   embedded quotation marks as well as embedded commas, as seen by
   case (d), above.

   Case (d) is the closest to Microsoft CSV format. *However*, the
   Microsoft CSV format allows embedded newlines within a
   double-quoted field. If embedded newlines within fields are a
   possibility for your data, you should consider using something
   other than sed to work with the data file.

   Before handling a comma-delimited data file, make sure that you
   fully understand its format and check the integrity of the data.
   Does each line contain the same number of fields? Should certain
   fields be composed only of numbers or of two-letter state
   abbreviations in all caps? Sed (or awk or perl) should be used to
   validate the integrity of the data file before you attempt to alter
   it or extract particular fields from the file.

   After ensuring that each line has a valid number of fields, use sed
   to locate and modify individual fields, using the \(...\) grouping
   command where needed.

   In case (a):

     sed 's/^[^,]*,[^,]*,[^,]*,[^,]*,/.../'
             ^     ^     ^
             |     |     |_ 3rd field
             |     |_______ 2nd field
             |_____________ 1st field

     # Unix script to delete the second field for case (a)
     sed 's/^\([^,]*\),[^,]*,/\1,,/' file

     # Unix script to change field 1 to 9999 for case (a)
     sed 's/^[^,]*,/9999,/' file

   In cases (b) and (c):

     sed 's/^"[^"]*","[^"]*","[^"]*","[^"]*",/.../'
              1st--   2nd--   3rd--   4th--

     # Unix script to delete the second field for case (c)
     sed 's/^\("[^"]*"\),"[^"]*",/\1,"",/' file

     # Unix script to change field 1 to 9999 for case (c)
     sed 's/^"[^"]*",/"9999",/' file


   In case (d):

   One way to parse such files is to replace the 3-character field
   separator "," with an unused character like the tab or vertical
   bar. (Technically, the field separator is only the comma while the
   fields are surrounded by "double quotes", but the net _effect_ is
   that fields are separated by quote-comma-quote, with quote
   characters added to the beginning and end of each record.) Search
   your datafile _first_ to make sure that your character appears
   nowhere in it!

     sed -n '/|/p' file        # search for any instance of '|'
     # if it's not found, we can use the '|' to separate fields

   Then replace the 3-character field separator and parse as before:

     # sed script to delete the second field for case (d)
     s/","/|/g;                  # global change of "," to bar
     s/^\([^|]*\)|[^|]|/\1||/;   # delete 2nd field
     s/|/","/g;                  # global change of bar back to ","
     #---end of script---

     # sed script to change field 1 to 9999 for case (d)
     # Remember to accommodate leading and trailing quote marks
     s/","/|/g;
     s/^[^|]*|/"9999|/;
     s/|/","/g;
     #---end of script---

   Note that this technique works only if _each_ and _every_ field is
   surrounded with double quotes, including empty fields.

   The following solution is for more complex examples of (d), such
   as: not all fields contain "double-quote" marks, or the presence of
   embedded "double-quote" marks within fields, or extraneous
   whitespace around field delimiters. (Thanks to Greg Ubben for this
   script!)

     # sed script to convert case (d) to bar-delimited records
     s/^ *\(.*[^ ]\) *$/|\1|/;
     s/" *, */"|/g;
     : loop
     s/| *\([^",|][^,|]*\) *, */|\1|/g;
     s/| *, */|\1|/g;
     t loop
     s/  *|/|/g;
     s/|  */|/g;
     s/^|\(.*\)|$/\1/;
     #---end of script---

   For example, it turns this (which is badly-formed but legal):

   first,"",unquoted ,""this" is, quoted " ,, sub "quote" inside, f", lone  " empty:

   into this:

   first|""|unquoted|""this" is, quoted "||sub "quote" inside|f"|lone  "   empty:

   Note that the script preserves the "double-quote" marks, but
   changes only the commas where they are used as field separators. I
   have used the vertical bar "|" because it's easier to read, but you
   may change this to another field separator if you wish.

   If your CSV datafile is more complex, it would probably not be
   worth the effort to write it in sed. For such a case, you should
   use Perl with a dedicated CSV module (there are at least two recent
   CSV parsers available from CPAN).

4.13. How do I handle fixed-length, columnar data?

   Sed handles fixed-length fields via \(grouping\) and backreferences
   (\1, \2, \3 ...). If we have 3 fields of 10, 25, and 9 characters
   per field, our sed script might look like so:

     s/^\(.\{10\}\)\(.\{25\}\)\(.\{9\}\)/\3\2\1/;  # Change the fields
        ^^^^^^^^^^^~~~~~~~~~~~==========           #   from 1,2,3 to 3,2,1
         field #1   field #2   field #3

   This is a bit hard to read. By using GNU sed or ssed with the -r
   switch active, it can look like this:

     s/^(.{10})(.{25})(.{9})/\3\2\1/;          # Using the -r switch

   To delete a field in sed, use grouping and omit the backreference
   from the field to be deleted. If the data is long or difficult to
   work with, use ssed with the -R switch and the /x flag after an s///
   command, to insert comments and remarks about the fields.

   For records with many fields, use GNU awk with the FIELDWIDTHS
   variable set in the top of the script. For example:

     awk 'BEGIN{FIELDWIDTHS = "10 25 9"}; {print $3 $2 $1}' file

   This is much easier to read than a similar sed script, especially
   if there are more than 5 or 6 fields to manipulate.

4.14. How do I commify a string of numbers?

   Use the simplest script necessary to accomplish your task. As
   variations of the line increase, the sed script must become more
   complex to handle additional conditions. Whole numbers are
   simplest, followed by decimal formats, followed by embedded words.

   Case 1: simple strings of whole numbers separated by spaces or
   commas, with an optional negative sign. To convert this:

       4381, -1222333, and 70000: - 44555666 1234567890 words
       56890  -234567, and 89222  -999777  345888777666 chars

   to this:

       4,381, -1,222,333, and 70,000: - 44,555,666 1,234,567,890 words
       56,890  -234,567, and 89,222  -999,777  345,888,777,666 chars

   use one of these one-liners:

     sed ':a;s/\B[0-9]\{3\}\>/,&/;ta'                      # GNU sed
     sed -e :a -e 's/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/;ta'  # other seds

   Case 2: strings of numbers which may have an embedded decimal
   point, separated by spaces or commas, with an optional negative
   sign. To change this:

       4381,  -6555.1212 and 70000,  7.18281828  44906982.071902
       56890   -2345.7778 and 8.0000:  -49000000 -1234567.89012

   to this:

       4,381,  -6,555.1212 and 70,000,  7.18281828  44,906,982.071902
       56,890   -2,345.7778 and 8.0000:  -49,000,000 -1,234,567.89012

   use the following command for GNU sed:

     sed ':a;s/\(^\|[^0-9.]\)\([0-9]\+\)\([0-9]\{3\}\)/\1\2,\3/g;ta'

   and for other versions of sed:

     sed -f case2.sed files

     # case2.sed
     s/^/ /;                 # add space to start of line
     :a
     s/\( [-0-9]\{1,\}\)\([0-9]\{3\}\)/\1,\2/g
     ta
     s/ //;                  # remove space from start of line
     #---end of script---

4.15. How do I prevent regex expansion on substitutions?

   Sometimes you want to *match* regular expression metacharacters as
   literals (e.g., you want to match "[0-9]" or "\n"), to be replaced
   with something else. The ordinary way to prevent expanding
   metacharacters is to prefix them with a backslash. Thus, if "\n"
   matches a newline, "\\n" will match the two-character string of
   'backslash' followed by 'n'.

   But doing this repeatedly can become tedious if there are many
   regexes. The following script will replace alternating strings of
   literals, where no character is interpreted as a regex
   metacharacter:

     # filename: sub_quote.sed
     #   author: Paolo Bonzini
     # sed script to add backslash to find/replace metacharacters
     N;                  # add even numbered line to pattern space
     s,[]/\\$*[],\\&,g;  # quote all of [, ], /, \, $, or *
     s,^,s/,;            # prepend "s/" to front of pattern space
     s,$,/,;             # append "/" to end of pattern space
     s,\n,/,;            # change "\n" to "/", making s/from/to/
     #---end of script---

   Here's a sample of how sub_quote.sed might be used. This example
   converts typical sed regexes to perl-style regexes. The input file
   consists of 10 lines:

       [0-9]
       \d
       [^0-9]
       \D
       \+
       +
       \?
       ?
       \|
       |

   Run the command "sed -f sub_quote.sed input", to transform the
   input file (above) to 5 lines of output:

       s/\[0-9\]/\\d/
       s/\[^0-9\]/\\D/
       s/\\+/+/
       s/\\?/?/
       s/\\|/|/

   The above file is itself a sed script, which can then be used to
   modify other files.

4.16. How do I convert a string to all lowercase or capital letters?

   The easiest method is to use a new version of GNU sed, ssed, sedmod
   or sed16 and employ the \U, \L, or other switches on the right side
   of an s/// command. For example, to convert any word which begins
   with "reg" or "exp" into solid capital letters:

       sed -r "s/\<(reg|exp)[a-z]+/\U&/g"              # gsed4.+ or ssed
       sed "s/\<reg[a-z]+/\U&/g; s/\<exp[a-z]+/\U&/g"  # sed16 and sedmod

   As you can see, sedmod and sed16 do not support alternation (|),
   but they do support case conversion. If none of these versions of
   sed are available to you, some sample scripts for this task are
   available from the Seder's Grab Bag:

       http://sed.sourceforge.net/grabbag/scripts

   Note that some case conversion scripts are listed under "Filename
   manipulation" and others are under "Text formatting."

   CHANGING BLOCKS (consecutive lines)

4.20. How do I change only one section of a file?

   You can match a range of lines by line number, by regexes (say, all
   lines between the words "from" and "until"), or by a combination of
   the two. For multiple substitutions on the same range, put the
   command(s) between braces {...}. For example:

     # replace only between lines 1 and 20
     1,20 s/Johnson/White/g

     # replace everywhere EXCEPT between lines 1 and 20
     1,20 !s/Johnson/White/g

     # replace only between words "from" and "until". Note the
     # use of \<....\> as word boundary markers in GNU sed.
     /from/,/until/ { s/\<red\>/magenta/g; s/\<blue\>/cyan/g; }

     # replace only from the words "ENDNOTES:" to the end of file
     /ENDNOTES:/,$ { s/Schaff/Herzog/g; s/Kraft/Ebbing/g; }

   For technical details on using address ranges, see section 3.3
   ("Addressing and Address ranges").

4.21. How do I delete or change a block of text if the block contains
      a certain regular expression?

   The following deletes the block between 'start' and 'end'
   inclusively, if and only if the block contains the string
   'regex'. Written by Russell Davies, with additional comments:

     # sed script to delete a block if /regex/ matches inside it
     :t
     /start/,/end/ {    # For each line between these block markers..
        /end/!{         #   If we are not at the /end/ marker
           $!{          #     nor the last line of the file,
              N;        #     add the Next line to the pattern space
              bt
           }            #   and branch (loop back) to the :t label.
        }               # This line matches the /end/ marker.
        /regex/d;       # If /regex/ matches, delete the block.
     }                  # Otherwise, the block will be printed.
     #---end of script---

   Note: When the script above reaches /regex/, the entire multi-line
   block is in the pattern space. To replace items inside the block,
   use "s///". To change the entire block, use the 'c' (change)
   command:

     /regex/c\
     1: This will replace the entire block\
     2: with these two lines of text.

4.22. How do I locate a paragraph of text if the paragraph contains a
      certain regular expression?

   Assume that paragraphs are separated by blank lines. For regexes
   that are single terms, use one of the following scripts:

     sed -e '/./{H;$!d;}' -e 'x;/regex/!d'      # most seds
     sed '/./{H;$!d;};x;/regex/!d'              # GNU sed

   To print paragraphs only if they contain 3 specific regular
   expressions (RE1, RE2, and RE3), in any order in the paragraph:
