Escape a string for a sed replace pattern

Question

In my bash script I have an external (received from user) string, which I should use in sed pattern.

REPLACE="<funny characters here>"
sed "s/KEYWORD/$REPLACE/g"

How can I escape the $REPLACE string so it would be safely accepted by sed as a literal replacement?

NOTE: The KEYWORD is a dumb substring with no matches etc. It is not supplied by user.

Are you trying to avoid the "Little Bobby Tables" problem if they say "/g -e 's/PASSWORD=.*/PASSWORD=abc/g'"? — Paul Tomblin, Jan 02 '09 at 17:53
If using bash, you don't need sed. Just use `outputvar="${inputvar//"$txt2replace"/"$txt2replacewith"}".` — Dennis Estenson, Dec 29 '15 at 23:00
@destenson I think you shouldn't be putting the two variables outside the quotes. Bash can read variables inside double-quotes (in your example, whitespace could screw things up). — Camilo Martin, Dec 30 '15 at 16:58
@CamiloMartin, see my comment on my own answer. The quotes inside of the ${} do not match up with the quotes inside. The two variables are *not* outside the quotes. — Dennis Estenson, Aug 18 '16 at 18:05
related: https://superuser.com/questions/422459/substitution-in-text-file-without-regular-expressions — Ciro Santilli OurBigBook.com, Mar 28 '18 at 13:05

score 364 · Accepted Answer · edited Oct 02 '21 at 05:25

364

Warning: This does not consider newlines. For a more in-depth answer, see this SO-question instead. (Thanks, Ed Morton & Niklas Peter)

Note that escaping everything is a bad idea. Sed needs many characters to be escaped to get their special meaning. For example, if you escape a digit in the replacement string, it will turn in to a backreference.

As Ben Blank said, there are only three characters that need to be escaped in the replacement string (escapes themselves, forward slash for end of statement and & for replace all):

ESCAPED_REPLACE=$(printf '%s\n' "$REPLACE" | sed -e 's/[\/&]/\\&/g')
# Now you can use ESCAPED_REPLACE in the original sed statement
sed "s/KEYWORD/$ESCAPED_REPLACE/g"

If you ever need to escape the KEYWORD string, the following is the one you need:

sed -e 's/[]\/$*.^[]/\\&/g'

And can be used by:

KEYWORD="The Keyword You Need";
ESCAPED_KEYWORD=$(printf '%s\n' "$KEYWORD" | sed -e 's/[]\/$*.^[]/\\&/g');

# Now you can use it inside the original sed statement to replace text
sed "s/$ESCAPED_KEYWORD/$ESCAPED_REPLACE/g"

Remember, if you use a character other than / as delimiter, you need replace the slash in the expressions above wih the character you are using. See PeterJCLaw's comment for explanation.

Edited: Due to some corner cases previously not accounted for, the commands above have changed several times. Check the edit history for details.

edited Oct 02 '21 at 05:25

Cadoiz

1,446
21
31

answered Apr 24 '10 at 18:35

Pianosaurus

5,538
2
20
15

33

It's worth noting that you can avoid having to escape the forward slashes by not using them as the delimiters. Most (all?) versions of sed allow you to use any character, so long as it fits the pattern: $ echo 'foo/bar' | sed s_/_:_ # foo:bar – PeterJCLaw Jun 18 '11 at 13:27
@PeterJCLaw: Good point. I believe that is true for all versions of sed. There are only two escaped slashes above, so it wouldn't make much difference, but it matters if you use another delimiter in the sed expression this output is inserted into. I added some info to reflect that. – Pianosaurus Jul 02 '11 at 10:43
This is amazing. Thanks... quick tweak for me (using this in a devops pipeline to replace a token with a client secret) I had to change the regex to 's/[\/&~]/\\&/g' (added a tilde character) as sed barfs when trying to replace a tild symbol.... our sed command is sed -i "s~##CLIENTSECRET##~$ESCAPED_SECRET~g" "$FILEPATH" which has tildes in it, so it makes sense. – Mark McGookin Nov 23 '22 at 14:32
Great solution and explanation guys. It seems to me you don't need to escape the closing bracket `]` ever (in your KEYWORD example). Just an idea if you're interested. Works in my case. Thank you. – George Pavelka Apr 01 '23 at 15:55
What is the need for a newline in printf? – Xavanteex Jun 14 '23 at 22:47
Escaping `]` isn't needed: https://unix.stackexchange.com/a/209744/101920 – mgutt Jul 28 '23 at 13:55

score 125 · Answer 2 · edited Jun 12 '16 at 18:31

125

The sed command allows you to use other characters instead of / as separator:

sed 's#"http://www\.fubar\.com"#URL_FUBAR#g'

The double quotes are not a problem.

edited Jun 12 '16 at 18:31

answered Apr 30 '15 at 11:40

scre_www

2,536
4
20
30

7

You still need to escape `.` which otherwise has a special meaning. I edited your answer. – ypid Jun 12 '16 at 17:54
I have just tried to do : `sed '/CLIENTSCRIPT="foo"/a CLIENTSCRIPT2="hello"' file` with `sed '|CLIENTSCRIPT="foo"|a CLIENTSCRIPT2="hello"' file` and that does not do the same. – Dimitri Kopriwa Nov 19 '18 at 13:34
1

Because this applies only to substitute, this should be saying: The `s` command (as in substitute) of sed allows you to use other characters instead of / as a separator. Also, this would be an answer to how to use sed on URL with slash characters. It does not answer the OP question how to escape a string entered by a user, which could contain /, \, but also # if you decide to use that. And besides, URI can contain # too – papo Dec 13 '18 at 16:54

score 53 · Answer 3 · edited Jun 01 '16 at 02:14

53

The only three literal characters which are treated specially in the replace clause are / (to close the clause), \ (to escape characters, backreference, &c.), and & (to include the match in the replacement). Therefore, all you need to do is escape those three characters:

sed "s/KEYWORD/$(echo $REPLACE | sed -e 's/\\/\\\\/g; s/\//\\\//g; s/&/\\\&/g')/g"

Example:

$ export REPLACE="'\"|\\/><&!"
$ echo fooKEYWORDbar | sed "s/KEYWORD/$(echo $REPLACE | sed -e 's/\\/\\\\/g; s/\//\\\//g; s/&/\\\&/g')/g"
foo'"|\/><&!bar

edited Jun 01 '16 at 02:14

Mark Stosberg

12,961
6
44
49

answered Jan 02 '09 at 18:31

Ben Blank

54,908
28
127
156

2

Also a newline, I think. How do I escape a newline? – Alexander Gladysh Jan 02 '09 at 18:52
2

Be careful what the default behavior of echo is with regard to backslashes. In bash, echo defaults to no interpretation of backslash escapes, which serves the purpose here. In dash (sh), on the other hand, echo interprets backslash escapes and has no way, as far as I know, of suppressing this. Therefore, in dash (sh), instead of echo $x, do printf '%s\n' $x. – Youssef Eldakar Dec 11 '12 at 08:43
Also, always use the -r option when doing a read to treat backslashes in user input as literals. – Youssef Eldakar Dec 11 '12 at 08:51
For cross-platform compatibility with other shells, you should consult this document regarding the replacement of sed special characters: http://www.grymoire.com/Unix/Sed.html#toc-uh-62 – Dejay Clayton Feb 15 '14 at 01:43
If only those three characters are treated specially, why does `echo '[a](!)' | sed 's/[a](!)/[a]/'` (with GNU sed inside bash) result in `[a](!)` instead of `[a]`? – Drux Aug 13 '16 at 20:11
2

@Drux The three characters are the only special ones in the *replace* clause. Much more is special in the pattern clause. – lenz May 08 '17 at 13:13
This saved my day, week, career, thank you!!! @Ben Blank – Ognjen Stanić Jan 19 '21 at 10:50
You can escape all of these characters and others with this more concise sed command: `echo $REPLACE | sed -e 's/[]\/$*.^[]/\\&/g'` The example above only escapes `\` `/` and `&`. But other characters can have special meaning too: `$` `*` `.` etc. We can make this escaping more concise by using `&` which just refers that portion of the pattern space which matched. – Chris Oct 25 '21 at 15:57

score 39 · Answer 4 · answered May 06 '12 at 01:46

39

Based on Pianosaurus's regular expressions, I made a bash function that escapes both keyword and replacement.

function sedeasy {
  sed -i "s/$(echo $1 | sed -e 's/\([[\/.*]\|\]\)/\\&/g')/$(echo $2 | sed -e 's/[\/&]/\\&/g')/g" $3
}

Here's how you use it:

sedeasy "include /etc/nginx/conf.d/*" "include /apps/*/conf/nginx.conf" /etc/nginx/nginx.conf

answered May 06 '12 at 01:46

Gurpartap Singh

2,744
1
26
30

4

thanks! if anyone else gets syntax error when trying to use it, just like me, just remember to run it using bash, not sh – Konstantin Pereiaslov Apr 08 '13 at 18:56
3

Is there a function just to escape a string for sed instead of wrapping around sed? – CMCDragonkai Jul 05 '14 at 02:28
1

Hey, just a general warning regarding starting pipes with an echo like this: Some (most?) implementations of echo take options (see `man echo`), causing the pipe to behave unexpectedly when your argument `$1` begins with a dash. Instead, you can start your pipe with `printf '%s\n' "$1"`. – Pianosaurus Mar 18 '19 at 13:59
1

It doesn't work with new lines e.g. " sedeasy "hello world" "hello\n world" "x.txt" – Mihai Oct 15 '21 at 12:17

score 20 · Answer 5 · edited Jun 09 '15 at 10:53

20

It's a bit late to respond... but there IS a much simpler way to do this. Just change the delimiter (i.e., the character that separates fields). So, instead of s/foo/bar/ you write s|bar|foo.

And, here's the easy way to do this:

sed 's|/\*!50017 DEFINER=`snafu`@`localhost`\*/||g'

The resulting output is devoid of that nasty DEFINER clause.

edited Jun 09 '15 at 10:53

fedorqui

275,237
103
548
598

answered Jan 23 '14 at 17:39

user2460464

475
4
5

12

No, `&` and `\` must still be escaped, as must the delimiter, whichever is chosen. – mirabilos May 07 '14 at 08:14
4

That solved my problem, as I had "/" chars in a replacement string. Thanks, man! – Evgeny Goldin Jun 04 '14 at 22:35
works for me. What am doing is try to escape `$` in the string about to be changed, and maintain the meaning of `$` in the replacement string. say I want to change `$XXX` to the value of variable `$YYY`, `sed -i "s|\$XXX|$YYY|g" file` works fine. – hakunami Nov 06 '14 at 08:25

Dennis Estenson · Answer 6 · 2016-08-18T18:07:27.237

13

It turns out you're asking the wrong question. I also asked the wrong question. The reason it's wrong is the beginning of the first sentence: "In my bash script...".

I had the same question & made the same mistake. If you're using bash, you don't need to use sed to do string replacements (and it's much cleaner to use the replace feature built into bash).

Instead of something like, for example:

function escape-all-funny-characters() { UNKNOWN_CODE_THAT_ANSWERS_THE_QUESTION_YOU_ASKED; }
INPUT='some long string with KEYWORD that need replacing KEYWORD.'
A="$(escape-all-funny-characters 'KEYWORD')"
B="$(escape-all-funny-characters '<funny characters here>')"
OUTPUT="$(sed "s/$A/$B/g" <<<"$INPUT")"

you can use bash features exclusively:

INPUT='some long string with KEYWORD that need replacing KEYWORD.'
A='KEYWORD'
B='<funny characters here>'
OUTPUT="${INPUT//"$A"/"$B"}"

edited Aug 18 '16 at 18:07

answered Dec 29 '15 at 22:51

Dennis Estenson

1,022
10
11

BTW, syntax highlighting here is wrong. The exterior quotes match up & the interior quotes match up. In other words, it looks like `$A` and `$B` are unquoted, but they are not. The quotes inside of the `${}` do not match with the quotes outside of it. – Dennis Estenson Jul 20 '16 at 17:37
2

You don't actually have to quote the right-hand side of an assignment (unless you want to do something like `var='has space'`) – `OUTPUT=${INPUT//"$A"/"$B"}` is safe. – Benjamin W. Jan 21 '17 at 20:27
You don't actually have to quote the right-hand side of an assignment (unless you want it to work in the real world and not just as a toy script to show yur mad skilz). I always try to quote every variable expansion that I don't want the shell to interpret, unless I have a specific reason not to. That way, things tend to break less often, especially when when provided with new or unexpected input. – Dennis Estenson Jan 21 '17 at 22:47
Just saying that an _expansion_ doesn't have to be quoted: `var1=$var2` will never break, no matter what you have in `var2`: glob characters, spaces, newlines... But quoting it doesn't hurt, of course. – Benjamin W. Jan 21 '17 at 22:52
1

See [manual](https://www.gnu.org/software/bash/manual/bashref.html#Shell-Parameters): "All values undergo tilde expansion, parameter and variable expansion, command substitution, arithmetic expansion, and quote removal (detailed below)." I.e., the same as in double quotes. – Benjamin W. Jan 24 '17 at 22:13
3

What if you need to use sed on a file? – Efren Aug 10 '18 at 07:48
There is a small mistake: the pattern ($A) needs double quotes but the replacement ($B) should be unqouted as otherwise the double quotes will become part of the replacement. Apart from that the perfect solution I was looking for - Thanks! – Jörg Apr 06 '21 at 16:41
new lines are broken – Mihai Oct 15 '21 at 13:11

score 1 · Answer 7 · edited Jun 09 '15 at 10:54

1

Use awk - it is cleaner:

$ awk -v R='//addr:\\file' '{ sub("THIS", R, $0); print $0 }' <<< "http://file:\_THIS_/path/to/a/file\\is\\\a\\ nightmare"
http://file:\_//addr:\file_/path/to/a/file\\is\\\a\\ nightmare

edited Jun 09 '15 at 10:54

fedorqui

275,237
103
548
598

answered Nov 14 '14 at 23:34

greggster

27
1

2

The trouble with `awk` is that it has nothing similar to `sed -i`, which comes extremely handy 99% of the time. – Tino Nov 27 '17 at 17:07
1

This is a step in the right direction, but awk still interprets some metacharacters in your substitution, so it's still not safe for user input. – Jeremy Huiskamp Jan 21 '19 at 14:53
@Tino : so u like having `sed` make nonstop stacks of highly duplicative backups instead of routing the `awk` output to a temp file name, verify it's the desired output, then rename it (or preferably, a safer OS-level operation like `cp clone` for `APFS`) – RARE Kpop Manifesto Aug 02 '22 at 22:10
@JeremyHuiskamp : `awk` interpret nearly nothing, not even `&`, for `OFS` and `ORS` (other than basics octals hex and compressing every pair of `\\ ` to `\ ` ), so that's one possible approach - i know it's weird, but `OFS` simply isn't a pure 1-to-1 mapping to using `gsub()`. if u're really really concerned about user input, one can always route it via a pseudo file `<( …. )` then those truly get ZERO interpretations – RARE Kpop Manifesto Aug 02 '22 at 22:14

score 0 · Answer 8 · answered Jan 02 '09 at 17:58

Here is an example of an AWK I used a while ago. It is an AWK that prints new AWKS. AWK and SED being similar it may be a good template.

ls | awk '{ print "awk " "'"'"'"  " {print $1,$2,$3} " "'"'"'"  " " $1 ".old_ext > " $1 ".new_ext"  }' > for_the_birds

It looks excessive, but somehow that combination of quotes works to keep the ' printed as literals. Then if I remember correctly the vaiables are just surrounded with quotes like this: "$1". Try it, let me know how it works with SED.

score 0 · Answer 9 · answered Apr 02 '20 at 20:56

0

These are the escape codes that I've found:

* = \x2a
( = \x28
) = \x29

" = \x22
/ = \x2f
\ = \x5c

' = \x27
? = \x3f
% = \x25
^ = \x5e

answered Apr 02 '20 at 20:56

Ark25

109
1
3

2

Not all `sed` dialects accept hex escapes with `\x`. There's not much to "discover"; you can look up character codes in any ASCII chart. – tripleee Jul 17 '20 at 17:05

Eduardo Lucio · Answer 10 · 2022-08-24T12:39:09.937

There are dozens of answers out there... If you don't mind using a bash function schema, below is a good answer. The objective below was to allow using sed with practically any parameter as a KEYWORD (F_PS_TARGET) or as a REPLACE (F_PS_REPLACE). We tested it in many scenarios and it seems to be pretty safe. The implementation below supports tabs, line breaks and sigle quotes for both KEYWORD and replace REPLACE.

NOTES: The idea here is to use sed to escape entries for another sed command.

CODE

F_REVERSE_STRING_R=""
f_reverse_string() {
    : 'Do a string reverse.

    To undo just use a reversed string as STRING_INPUT.

    Args:
        STRING_INPUT (str): String input.

    Returns:
        F_REVERSE_STRING_R (str): The modified string.
    '

    local STRING_INPUT=$1
    F_REVERSE_STRING_R=$(echo "x${STRING_INPUT}x" | tac | rev)
    F_REVERSE_STRING_R=${F_REVERSE_STRING_R%?}
    F_REVERSE_STRING_R=${F_REVERSE_STRING_R#?}
}

# [Ref(s).: https://stackoverflow.com/a/2705678/3223785 ]
F_POWER_SED_ECP_R=""
f_power_sed_ecp() {
    : 'Escape strings for the "sed" command.

    Escaped characters will be processed as is (e.g. /n, /t ...).

    Args:
        F_PSE_VAL_TO_ECP (str): Value to be escaped.
        F_PSE_ECP_TYPE (int): 0 - For the TARGET value; 1 - For the REPLACE value.

    Returns:
        F_POWER_SED_ECP_R (str): Escaped value.
    '

    local F_PSE_VAL_TO_ECP=$1
    local F_PSE_ECP_TYPE=$2

    # NOTE: Operational characters of "sed" will be escaped, as well as single quotes.
    # By Questor
    if [ ${F_PSE_ECP_TYPE} -eq 0 ] ; then
    # NOTE: For the TARGET value. By Questor

        F_POWER_SED_ECP_R=$(echo "x${F_PSE_VAL_TO_ECP}x" | sed 's/[]\/$*.^[]/\\&/g' | sed "s/'/\\\x27/g" | sed ':a;N;$!ba;s/\n/\\n/g')
    else
    # NOTE: For the REPLACE value. By Questor

        F_POWER_SED_ECP_R=$(echo "x${F_PSE_VAL_TO_ECP}x" | sed 's/[\/&]/\\&/g' | sed "s/'/\\\x27/g" | sed ':a;N;$!ba;s/\n/\\n/g')
    fi

    F_POWER_SED_ECP_R=${F_POWER_SED_ECP_R%?}
    F_POWER_SED_ECP_R=${F_POWER_SED_ECP_R#?}
}

# [Ref(s).: https://stackoverflow.com/a/24134488/3223785 ,
# https://stackoverflow.com/a/21740695/3223785 ,
# https://unix.stackexchange.com/a/655558/61742 ,
# https://stackoverflow.com/a/11461628/3223785 ,
# https://stackoverflow.com/a/45151986/3223785 ,
# https://linuxaria.com/pills/tac-and-rev-to-see-files-in-reverse-order ,
# https://unix.stackexchange.com/a/631355/61742 ]
F_POWER_SED_R=""
f_power_sed() {
    : 'Facilitate the use of the "sed" command. Replaces in files and strings.

    Args:
        F_PS_TARGET (str): Value to be replaced by the value of F_PS_REPLACE.
        F_PS_REPLACE (str): Value that will replace F_PS_TARGET.
        F_PS_FILE (Optional[str]): File in which the replacement will be made.
        F_PS_SOURCE (Optional[str]): String to be manipulated in case "F_PS_FILE" was
    not informed.
        F_PS_NTH_OCCUR (Optional[int]): [1~n] - Replace the nth match; [n~-1] - Replace
    the last nth match; 0 - Replace every match; Default 1.

    Returns:
        F_POWER_SED_R (str): Return the result if "F_PS_FILE" is not informed.
    '

    local F_PS_TARGET=$1
    local F_PS_REPLACE=$2
    local F_PS_FILE=$3
    local F_PS_SOURCE=$4
    local F_PS_NTH_OCCUR=$5
    if [ -z "$F_PS_NTH_OCCUR" ] ; then
        F_PS_NTH_OCCUR=1
    fi

    local F_PS_REVERSE_MODE=0
    if [ ${F_PS_NTH_OCCUR} -lt -1 ] ; then
        F_PS_REVERSE_MODE=1
        f_reverse_string "$F_PS_TARGET"
        F_PS_TARGET="$F_REVERSE_STRING_R"
        f_reverse_string "$F_PS_REPLACE"
        F_PS_REPLACE="$F_REVERSE_STRING_R"
        f_reverse_string "$F_PS_SOURCE"
        F_PS_SOURCE="$F_REVERSE_STRING_R"
        F_PS_NTH_OCCUR=$((-F_PS_NTH_OCCUR))
    fi

    f_power_sed_ecp "$F_PS_TARGET" 0
    F_PS_TARGET=$F_POWER_SED_ECP_R
    f_power_sed_ecp "$F_PS_REPLACE" 1
    F_PS_REPLACE=$F_POWER_SED_ECP_R

    local F_PS_SED_RPL=""
    if [ ${F_PS_NTH_OCCUR} -eq -1 ] ; then
    # NOTE: We kept this option because it performs better when we only need to replace
    # the last occurrence. By Questor

        # [Ref(s).: https://linuxhint.com/use-sed-replace-last-occurrence/ ,
        # https://unix.stackexchange.com/a/713866/61742 ]
        F_PS_SED_RPL="'s/\(.*\)$F_PS_TARGET/\1$F_PS_REPLACE/'"
    elif [ ${F_PS_NTH_OCCUR} -gt 0 ] ; then
        # [Ref(s).: https://unix.stackexchange.com/a/587924/61742 ]
        F_PS_SED_RPL="'s/$F_PS_TARGET/$F_PS_REPLACE/$F_PS_NTH_OCCUR'"
    elif [ ${F_PS_NTH_OCCUR} -eq 0 ] ; then
        F_PS_SED_RPL="'s/$F_PS_TARGET/$F_PS_REPLACE/g'"
    fi

    # NOTE: As the "sed" commands below always process literal values for the "F_PS_TARGET"
    # so we use the "-z" flag in case it has multiple lines. By Quaestor
    # [Ref(s).: https://unix.stackexchange.com/a/525524/61742 ]
    if [ -z "$F_PS_FILE" ] ; then
        F_POWER_SED_R=$(echo "x${F_PS_SOURCE}x" | eval "sed -z $F_PS_SED_RPL")
        F_POWER_SED_R=${F_POWER_SED_R%?}
        F_POWER_SED_R=${F_POWER_SED_R#?}
        if [ ${F_PS_REVERSE_MODE} -eq 1 ] ; then
            f_reverse_string "$F_POWER_SED_R"
            F_POWER_SED_R="$F_REVERSE_STRING_R"
        fi
    else
        if [ ${F_PS_REVERSE_MODE} -eq 0 ] ; then
            eval "sed -i -z $F_PS_SED_RPL \"$F_PS_FILE\""
        else
            tac "$F_PS_FILE" | rev | eval "sed -z $F_PS_SED_RPL" | tac | rev > "$F_PS_FILE"
        fi
    fi

}

MODEL

f_power_sed "F_PS_TARGET" "F_PS_REPLACE" "" "F_PS_SOURCE"
echo "$F_POWER_SED_R"

EXAMPLE

f_power_sed "{ gsub(/,[ ]+|$/,\"\0\"); print }' ./  and eliminate" "[ ]+|$/,\"\0\""  "" "Great answer (+1). If you change your awk to awk '{ gsub(/,[ ]+|$/,\"\0\"); print }' ./  and eliminate that concatenation of the final \", \" then you don't have to go through the gymnastics on eliminating the final record. So: readarray -td '' a < <(awk '{ gsub(/,[ ]+/,\"\0\"); print; }' <<<\"$string\") on Bash that supports readarray. Note your method is Bash 4.4+ I think because of the -d in readar"
echo "$F_POWER_SED_R"

IF YOU JUST WANT TO ESCAPE THE PARAMETERS TO THE SED COMMAND

MODEL

# "TARGET" value.
f_power_sed_ecp "F_PSE_VAL_TO_ECP" 0
echo "$F_POWER_SED_ECP_R"

# "REPLACE" value.
f_power_sed_ecp "F_PSE_VAL_TO_ECP" 1
echo "$F_POWER_SED_ECP_R"

IMPORTANT: If the strings for KEYWORD and/or replace REPLACE contain tabs or line breaks you will need to use the "-z" flag in your "sed" command. More details here.

EXAMPLE

f_power_sed_ecp "{ gsub(/,[ ]+|$/,\"\0\"); print }' ./  and eliminate" 0
echo "$F_POWER_SED_ECP_R"
f_power_sed_ecp "[ ]+|$/,\"\0\"" 1
echo "$F_POWER_SED_ECP_R"

NOTE: The f_power_sed_ecp and f_power_sed functions above was made available completely free as part of this project ez_i - Create shell script installers easily!.

Please read [correct-bash-and-shell-script-variable-capitalization](https://stackoverflow.com/questions/673055/correct-bash-and-shell-script-variable-capitalization) and [why-should-eval-be-avoided-in-bash-and-what-should-i-use-instead](https://stackoverflow.com/questions/17529220/why-should-eval-be-avoided-in-bash-and-what-should-i-use-instead). — Ed Morton, Aug 06 '22 at 11:18

score 0 · Answer 11 · answered Aug 02 '22 at 22:05

sed is typically a mess, especially the difference between gnu-sed and bsd-sed

might just be easier to place some sort of sentinel at the sed side, then a quick pipe over to awk, which is far more flexible in accepting any ERE regex, escaped hex, or escaped octals.

e.g. OFS in awk is the true replacement ::

date | sed -E 's/[0-9]+/\xC1\xC0/g' |  

          mawk NF=NF FS='\xC1\xC0' OFS='\360\237\244\241'

 1  Tue Aug   :: EDT

(tested and confirmed working on both BSD-sed and GNU-sed - the emoji isn't a typo that's what those 4 bytes map to in UTF-8 )

score 0 · Answer 12 · answered Oct 24 '22 at 17:08

0

Standard recommendation here: use perl :)

echo KEYWORD > /tmp/test

REPLACE="<funny characters here>"
perl -pi.bck -e "s/KEYWORD/${REPLACE}/g" /tmp/test
cat /tmp/test

answered Oct 24 '22 at 17:08

Dzmitry Sankouski

160
1
11

score -1 · Answer 13 · answered Nov 22 '13 at 15:29

-1

don't forget all the pleasure that occur with the shell limitation around " and '

so (in ksh)

Var=">New version of \"content' here <"
printf "%s" "${Var}" | sed "s/[&\/\\\\*\\"']/\\&/g' | read -r EscVar

echo "Here is your \"text\" to change" | sed "s/text/${EscVar}/g"

answered Nov 22 '13 at 15:29

NeronLeVelu

9,908
1
23
43

exactly the direction I was needed, for escaping find results, found through google so may be helpful for someone - ended with - sed "s/[&\\\\*\\"\'\"' )(]/\\&/g' – MolbOrg Oct 12 '18 at 18:31

score -2 · Answer 14 · answered Jun 01 '16 at 02:24

If the case happens to be that you are generating a random password to pass to sed replace pattern, then you choose to be careful about which set of characters in the random string. If you choose a password made by encoding a value as base64, then there is is only character that is both possible in base64 and is also a special character in sed replace pattern. That character is "/", and is easily removed from the password you are generating:

# password 32 characters log, minus any copies of the "/" character.
pass=`openssl rand -base64 32 | sed -e 's/\///g'`;

score -2 · Answer 15 · edited Sep 01 '17 at 06:38

-2

If you are just looking to replace Variable value in sed command then just remove Example:

sed -i 's/dev-/dev-$ENV/g' test to sed -i s/dev-/dev-$ENV/g test

edited Sep 01 '17 at 06:38

The Hungry Dictator

3,444
5
37
53

answered Sep 01 '17 at 06:35

Shailender Singh

27
1

score -2 · Answer 16 · edited Sep 30 '17 at 21:27

-2

I have an improvement over the sedeasy function, which WILL break with special characters like tab.

function sedeasy_improved {
    sed -i "s/$(
        echo "$1" | sed -e 's/\([[\/.*]\|\]\)/\\&/g' 
            | sed -e 's:\t:\\t:g'
    )/$(
        echo "$2" | sed -e 's/[\/&]/\\&/g' 
            | sed -e 's:\t:\\t:g'
    )/g" "$3"
}

So, whats different? $1 and $2 wrapped in quotes to avoid shell expansions and preserve tabs or double spaces.

Additional piping | sed -e 's:\t:\\t:g' (I like : as token) which transforms a tab in \t.

edited Sep 30 '17 at 21:27

Axel

3,331
11
35
58

answered Sep 30 '17 at 19:41

Francisco De Zuviria

19
4

But see my comment on the sedeasy answer regarding using echo in pipes. – Pianosaurus Mar 18 '19 at 14:02
Piping `sed` to `sed` is just silly; a single `sed` instance can execute an arbitrarily long and complex script. – tripleee Jul 17 '20 at 17:08

score -7 · Answer 17 · answered Oct 06 '17 at 06:54

-7

An easier way to do this is simply building the string before hand and using it as a parameter for sed

rpstring="s/KEYWORD/$REPLACE/g"
sed -i $rpstring  test.txt

answered Oct 06 '17 at 06:54

Javonne Martin

76
4

Fails and extremely dangerous, as REPLACE is user supplied: `REPLACE=/` gives ``sed: -e expression #1, char 12: unknown option to `s'`` – Tino Nov 27 '17 at 17:02

Escape a string for a sed replace pattern

17 Answers17

Linked

Related