Python Extension Proposal 498: Literal String Formatting

[asgard1024]

I am against it, because it allows arbitrary Python expressions inside format strings. It's too complicated and lets user have two different ways of doing things (not Pythonic) - one to calculate expression inside string and the other to calculate it outside (which should IMHO be preferred). This should maybe go to the standard library, but please not into the language.

I think a better approach would be to just add special formatter operators (if they aren't already there) that would just call str() or repr() or ascii() to whatever is presented to them (and maybe take some optional arguments such as length or padding).

[Camillo]

> I am against it, because it allows arbitrary Python expressions inside format strings.

As well it should. Programming language features should be orthogonal as much as possible.

> It's too complicated and lets user have two different ways of doing things (not Pythonic) - one to calculate expression inside string and the other to calculate it outside (which should IMHO be preferred).

You must hate expression nesting, then. Look at all these ways of doing the same thing:

    x = a + b * (c - d)

    e = c - d
    x = a + b * e

    f = b * (c - d)
    x = a + f

    e = c - d
    f = b * e
    x = a + f

Clearly, expressions should be restricted to no more than one binary operator. That reduces the number of different ways of computing the expression, and forces the programmer to give a name to each sub-step, which enhances readability, clarity, and debugging-friendliness.

I feel dirty just for having written that.

[task_queue]

> I am against it, because it allows arbitrary Python expressions inside format strings.

I was for this PEP until your post made this reality apparent. I'll take security over convenience.

[baq]

disagree. it only allows Python expressions in the same context in which they are already allowed: in the code. it's not possible to create an f-string at runtime.

[digisign]

It's an industry std now, can't put it in a lib without extra syntax.

[Walkman]

    There should be one-- and preferably only one --obvious way to do it.

This would be the 4th way of formatting strings in Python.

[chrismorgan]

It would also render the previous ways obsolete. New-style formatting never took over completely from old-style because it wasn’t sufficiently compelling—`"%s %s" % (a, b)` versus `"{} {}".format(a, b)` doesn’t have a clear winner. But `f"{a} {b}"`? Clearly superior. With the exception of backwards compatibility matters (which will be a nuisance for far too long), there would really be no reason to keep using the old ways in most places. i18n/l10n would really be the only mainstream reason for using anything other than f-strings.

[JoshTriplett]

Which clearly implies that the first three were insufficiently obvious. Or insufficiently Dutch.

[ceronman]

    Although practicality beats purity.

[baq]

it'd be the first one that's safe* in runtime.

*obligatory "not really, but in most cases" disclaimer

[declnz]

I would be very happy to see some version of this accepted.

When it comes to native String interpolation Groovy has it, Scala has it, ES6 has it apparently; to a more limited extent Bash, PHP, Perl have it too of course.

I can't help feeling that other devs are now coming to Python expecting this kind of feature, and are disappointed to find three (harder, often less readable) ways to do it instead. Got to keep up with the Joneses, etc...

[svisser]

It doesn't mean this is a good feature to have - it allows objects within scope to now be directly included in a string, which isn't a secure thing to do.

[schmichael]

Nonononono. Python does not need more string literal specifiers. For a language that avoid symbols and sigils like the plague, it already has an absurd amount of string literal syntax.

Why not make a strfmt library on pypi that provides a single fmt(s, args, kwargs) function and let people call that? Why the obsession with more builtins?

[mangeletti]

The first thing I thought of when I looked at the PEP was, "this is like a string version of register_globals=on".

A string literal whose value automatically changes with the code surrounding it sounds like a really bad idea.

I also noticed that the PEP uses str.format method as a strawman, ignoring the fact that % string interpolation is very popular and does not need replacing, which is at the core of this problem in the first place; Someone keeps trying to replace something that does not need replacing.

Furthermore, I can't help but think that this would eventually become a complete literal string DSL (if not one already) inside of Python.

I hope this PEP does not get accepted.

[TazeTSchnitzel]

> The first thing I thought of when I looked at the PEP was, "this is like a string version of register_globals=on", which is an unsettling thought to have about Python, my favorite language.

> The idea of having a string that is automatically dynamic and whose value is hardly predictable upon first glance, wholly dependent on the stability of the code surrounding it, sounds like an absolutely horrendous idea.

What?! It's not a dynamic string. It's a string concatenation expression with syntactic sugar.

This isn't PHP's register_globals. It's PHP's "{$n + 1}".

What this does, you can already do. "Foo " + bar + " baz" already exists. This is merely nicer syntax.

[JoshTriplett]

This is a massive improvement. I currently use .format(), sometimes with locals(), but f-strings would improve this massively.

Now if only they didn't require Python 3, so I could use them on the production systems I'm working on...

[ant6n]

I switch back and forth between format and %, and never use locals in the format. It's annoying, every time a string is written, to try to decide which way is better for this instance. That said, is there a way to do ('%.3f' % x) with this?

[dalke]

I am having some problems trying to understand the implementation. What would the AST from evaluating "f'{a+1}'" look like? Will there be a special AST node for f-strings, or will it be pre-structured into the AST?

If it's a special node, is it the responsibility of the byte code generator to parse the string? My belief is that it's part of the parser's job, so the AST will never contain an f-string.

What does a syntax error report look like? Or traceback? Will it be able to narrow down the part of the string which causes a problem?

Can f-strings include f-strings, like:

    f"{a + (f' and {b+1}')}"

I assume the answer is 'yes, and you shouldn't do that', which I can accept.

Support for arbitrary expressions inside of an f-string means that the following is allowed,

    def f(a, b):
        return f"{yield a} = {b}"

    print(f(1, 2))

and will work, and will print something, but it won't be "1 = 2". Nor will any but heavy-weight analysis tools be able to figure out that this 'f' is a generator.

I am less happy accepting that a magic string can turn a function into a generator. Take for example this code from around line 438 of https://searchcode.com/codesearch/view/18830026/ :

                if attr=='yields' :
                    yield_unit = self._grab_attr_(obj,'yield_unit')
                    if yield_unit:
                        ret = '%s %s'%(ret,yield_unit) # FIXME: i18n?
                return ret

The penultimate line could be rewritten, validly, as:

                        ret = f'{ret} {yield_unit}' # FIXME: i18n?

The introduction of a typo, from 'yield_unit' to 'yield unit', would drastically change the function, and be very hard to spot.

                        ret = f'{ret} {yield unit}' # FIXME: i18n?

Yes, Don't Do That, but we know that people use things like syntax highlighters to help understand the code and identify mistakes like this.

EDIT: the PEP says that the expression are "parsed with the equivalent of ast.parse(expression, '<fstring>', 'eval')". That means that 'yield' is not allowed.

[TazeTSchnitzel]

Presumably it'd be handled similarly to how PHP handles {} and $ in strings. As soon as possible, you swap "{foo} {bar}" for (foo.__format__() + " " + bar.__format__())

[erikb]

One of those things when you read it you wonder why it wasn't done that way in the first place.

[imakesnowflakes]

Exactly when you need to apply this principle...

https://en.wikipedia.org/wiki/Wikipedia:Chesterton's_fence

>In the matter of reforming things, as distinct from deforming them, there is one plain and simple principle; a principle which will probably be called a paradox. There exists in such a case a certain institution or law; let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, “I don’t see the use of this; let us clear it away.” To which the more intelligent type of reformer will do well to answer: “If you don’t see the use of it, I certainly won’t let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it

[jonathaneunice]

For those that want something close today, there's https://pypi.python.org/pypi/say

    fmt("Hello, {name}! You have {len(msgs)} waiting.")

Interpolates local variables and expressions. It uses the format method, and has all of format's output formatting.

[RubyPinch]

say looks nice, but it still suffers from exactly what this pep is trying to fix

    >>> def test(x=1):return lambda:say.say("{x}")
    >>> test()
    NameError: name 'x' is not defined

https://github.com/syrusakbary/interpy seems like a closer solution, could be modified to also support format specs

[kazinator]

> * fmt("Hello, {name}! You have {len(msgs)} waiting.")*

Yikes!

Please don't tell me that's a function which peeks at the caller's lexical variables, at run time, by name?

I see "inspect.currentframe().f_back" hacks in the code, good grief.

[voyou]

There's an interesting competing PEP which allows for the way in which the expressions are interpolated into the string to be customized: https://www.python.org/dev/peps/pep-0501/

[ant6n]

I couldn't get myself to read beyond this example

  i"Substitute $names and ${expressions} at runtime"

It looks like bash.

[ceronman]

C# 6.0 is also adding literal string formatting: http://www.codeproject.com/Articles/846566/What-s-new-in-Csh...

[stinos]

I hope this gets implemented. A year ago or so we had several customers wanting to custom format output filenames and directories in a desktop application and we settled for something which is almost exactly this, so the {identifier:format spec} idea, and ever since implementing it I wish any language had it as we found it really convenient and having no apparant disadvantages in comparision with printf-%-style in C or Python/streams in C++/{}-style in C#/Python

[chrismorgan]

I don’t understand what you’re seeking. This is purely for string literals. If you’re taking a user input string like `"foo-{date}.{extension}"`, you could use `string.format(date=…, extension=…)`

[CmonDev]

C# has it.

[RodericDay]

However, str.format() is not without its issues. Chief among them is its verbosity. For example, the text 'value' is repeated here:

    >>> value = 4 * 20
    >>> 'The value is {value}.'.format(value=value)
    'The value is 80.'

Even in its simplest form, there is a bit of boilerplate, and the value that's inserted into the placeholder is sometimes far removed from where the placeholder is situated:

    >>> 'The value is {}.'.format(value)
    'The value is 80.'

With an f-string, this becomes:

    >>> f'The value is {value}.'
    'The value is 80.'

Yeah I've had this thought before.

[Retra]

    >>> 'The value is '+value+'.'
    'The value is 80.'

Boilerplate? It's one extra character.

[publicfig]

In regards to the title, PEP stands for "Python Enhancement Proposal", not "Python Extension Proposal"

[deniska]

You can even hack it into a string class if you don't mind using even more scary hacks like monkey patching built in classes.

    def I(s):
        import inspect
        frame = inspect.currentframe()
        caller_locals = frame.f_back.f_locals
        return s.format(**caller_locals)

    def main():
        a = 12
        b = 10
        print I('A is {a} and B is {b}')

    if __name__ == '__main__':
        main()

[kazinator]

In TXR Lisp, from system prompt:

  # simple quasiliteral, denoted by backticks
  $ txr -p '(let ((x "Bob")) `Hello, @{x -10}`)'
  "Hello,        Bob"

  # word-quasiliteral (breaks into list on spaces)
  # denoted by hash-backtick:
  $ txr -p '#`@(+ 2 2) @(+ 1 2) a b c`'
  ("4" "3" "a" "b" "c")

  # op-argument references in quasiliteral
  # ret produces a function whose arguments depend
  # on the uses of @1, @2. ... and @rest in the
  # expression. The value of the expression is returned.
  # These references can emanate from a quasistring:

  $ txr -p "(mapcar (ret \`@1--@2--@rest\`) '(1 2 3) '(a b c) '(x y z)))"
  ("1--a--x" "2--b--y" "3--c--z")

  # Very basic indexing, slicing and field adjustment:
  $ txr -p '`foo @{(list 1 2 3) [0]} bar`'
  "foo 1 bar"
  $ txr -p '`foo @{(list 1 2 3) [2]} bar`'
  "foo 3 bar"
  $ txr -p '`foo @{(list 1 2 3) [1..:]} bar`'
  "foo 2 3 bar"
  $ txr -p '`foo @{(list 1 2 3) [1..:] 20} bar`'
  "foo 2 3                  bar"
  $ txr -p '`foo @{(list 1 2 3) [1..:] -20} bar`'
  "foo                  2 3 bar"

Interpolation into quasi-literals is very useful and expressive; I use it all the time. Doing trivial things should look trivial in the code.

Oh, right: referencing splices and unquotes is possible from inside a quasistring:

  $ txr -p '(let ((a 42) (b (range 1 5)))
             ^(list `hey @,a @(list ,*b)`))'
  (list (sys:quasi "hey "
          @42 " " @(list 1 2 3 4 5)))

  $ txr -p '(eval (let ((a 42) (b (range 1 5)))
                    ^(list `hey @,a @(list ,*b)`)))'
  ("hey 42 1 2 3 4 5")

Safe to say, that one's not coming to a Python near you.

[currywurst]

This looks like template strings in ES6 [1], which is really well done ! I look forward to using f-strings

[1]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

[TazeTSchnitzel]

This is a very handy feature of PHP and would be useful in Python. I think readability will be solved by syntax highlighting: expressions in an f-string would be highlighted like normal expressions, rather than like string content. This is what is already done for PHP.

[brazzledazzle]

I'm really happy to see this. I know it's petty, but this was the biggest reason I decided to focus on learning the ins and outs of Ruby instead of Python.

[riffraff]

I understand the need for expressions-in-literal.

I really don't understand why the unnecessary extra "!rsa" modifiers are a good thing though.

[TazeTSchnitzel]

The fact they had to hack in a workaround so != works is a point against it. And they acknowledge you can use repr()/str()/ascii() directly.

They want to keep it for str.format() compatibility, but I'm unconvinced. It hurts readability, and is redundant (There should be one-- and preferably only one --obvious way to do it.)