Python, being a beautifully designed high-level and interpreter-based programming language,
provides us with many features for the programmer's comfort.
But sometimes, the outcomes of a Python snippet may not seem obvious at first sight.
Here's a fun project attempting to explain what exactly is happening under the hood for some counter-intuitive snippets
and lesser-known features in Python.
While some of the examples you see below may not be WTFs in the truest sense,
but they'll reveal some of the interesting parts of Python that you might be unaware of.
I find it a nice way to learn the internals of a programming language, and I believe that you'll find it interesting too!
If you're an experienced Python programmer, you can take it as a challenge to get most of them right in the first attempt
You may have already experienced some of them before, and I might be able to revive sweet old memories of yours! :sweat_smile:
PS: If you're a returning reader, you can learn about the new modifications here
(the examples marked with asterisk are the ones added in the latest major revision).
(Optional): One line describing the unexpected output.
💡 Explanation:
Brief explanation of what's happening and why is it happening.
# Set up code
# More examples for further clarification (if necessary)
Output (Python version(s)):
>>> trigger # some example that makes it easy to unveil the magic
# some justified output
Note: All the examples are tested on Python 3.5.2 interactive interpreter,
and they should work for all the Python versions unless explicitly specified before the output.
Usage
A nice way to get the most out of these examples, in my opinion, is to read them in sequential order, and for every example:
Carefully read the initial code for setting up the example.
If you're an experienced Python programmer, you'll successfully anticipate what's going to happen next most of the time.
Read the output snippets and,
Check if the outputs are the same as you'd expect.
Make sure if you know the exact reason behind the output being the way it is.
If the answer is no (which is perfectly okay), take a deep breath, and read the explanation
(and if you still don't understand, shout out! and create an issue here).
If yes, give a gentle pat on your back, and you may skip to the next example.
👀 Examples
Section: Strain your brain!
▶ First things first! *
For some reason, the Python 3.8's "Walrus" operator (:=) has become quite popular. Let's check it out,
1.
# Python version 3.8+
>>> a = "wtf_walrus"
>>> a
'wtf_walrus'
>>> a := "wtf_walrus"
File "<stdin>", line 1
a := "wtf_walrus"
^
SyntaxError: invalid syntax
>>> (a := "wtf_walrus") # This works though
'wtf_walrus'
>>> a
'wtf_walrus'
2 .
# Python version 3.8+
>>> a = 6, 9
>>> a
(6, 9)
>>> (a := 6, 9)
(6, 9)
>>> a
6
>>> a, b = 6, 9 # Typical unpacking
>>> a, b
(6, 9)
>>> (a, b = 16, 19) # Oops
File "<stdin>", line 1
(a, b = 16, 19)
^
SyntaxError: invalid syntax
>>> (a, b := 16, 19) # This prints out a weird 3-tuple
(6, 16, 19)
>>> a # a is still unchanged?
6
>>> b
16
💡 Explanation
The Walrus operator (:=) was introduced in Python 3.8,
it can be useful in situations where you'd want to assign values to variables within an expression.
def some_func():
# Assume some expensive computation here
# time.sleep(1000)
return 5
# So instead of,
if some_func():
print(some_func()) # Which is bad practice since computation is happening twice
# or
a = some_func()
if a:
print(a)
# Now you can concisely write
if a := some_func():
print(a)
Output (> 3.8):
5
5
5
This saved one line of code, and implicitly prevented invoking some_func twice.
Unparenthesized "assignment expression" (use of walrus operator), is restricted at the top level,
hence the SyntaxError in the a := "wtf_walrus" statement of the first snippet.
Parenthesizing it worked as expected and assigned a.
As usual, parenthesizing of an expression containing = operator is not allowed.
Hence the syntax error in (a, b = 6, 9).
The syntax of the Walrus operator is of the form NAME:= expr, where NAME is a valid identifier,
and expr is a valid expression. Hence, iterable packing and unpacking are not supported which means,
(a := 6, 9) is equivalent to ((a := 6), 9) and ultimately (a, 9) (where a's value is 6')
>>> (a := 6, 9) == ((a := 6), 9)
True
>>> x = (a := 696, 9)
>>> x
(696, 9)
>>> x[0] is a # Both reference same memory location
True
Similarly, (a, b := 16, 19) is equivalent to (a, (b := 16), 19) which is nothing but a 3-tuple.
2. True because it is invoked in script. Might be False in python shell or ipython
a = "wtf"
b = "wtf"
assert a is b
a = "wtf!"
b = "wtf!"
assert a is b
3. True because it is invoked in script. Might be False in python shell or ipython
a, b = "wtf!", "wtf!"
assert a is b
a = "wtf!"; b = "wtf!"
assert a is b
4. Disclaimer - snippet is not relevant in modern Python versions
Output (< Python3.7 )
>>> 'a' * 20 is 'aaaaaaaaaaaaaaaaaaaa'
True
>>> 'a' * 21 is 'aaaaaaaaaaaaaaaaaaaaa'
False
Makes sense, right?
💡 Explanation:
The behavior in first and second snippets is due to a CPython optimization (called string interning)
that tries to use existing immutable objects in some cases rather than creating a new object every time.
After being "interned," many variables may reference the same string object in memory (saving memory thereby).
In the snippets above, strings are implicitly interned. The decision of when to implicitly intern a string is
implementation-dependent. There are some rules that can be used to guess if a string will be interned or not:
All length 0 and length 1 strings are interned.
Strings are interned at compile time ('wtf' will be interned but ''.join(['w', 't', 'f']) will not be interned)
Strings that are not composed of ASCII letters, digits or underscores, are not interned.
This explains why 'wtf!' was not interned due to !. CPython implementation of this rule can be found here
When a and b are set to "wtf!" in the same line, the Python interpreter creates a new object,
then references the second variable at the same time. If you do it on separate lines, it doesn't "know" that
there's already "wtf!" as an object (because "wtf!" is not implicitly interned as per the facts mentioned above).
It's a compile-time optimization. This optimization doesn't apply to 3.7.x versions of CPython
(check this issue for more discussion).
A compile unit in an interactive environment like IPython consists of a single statement,
whereas it consists of the entire module in case of modules. a, b = "wtf!", "wtf!" is single statement,
whereas a = "wtf!"; b = "wtf!" are two statements in a single line.
This explains why the identities are different in a = "wtf!"; b = "wtf!",
and also explain why they are same when invoked in some_file.py
The abrupt change in the output of the fourth snippet is due to a
peephole optimization technique known as Constant folding.
This means the expression 'a'*20 is replaced by 'aaaaaaaaaaaaaaaaaaaa' during compilation to save
a few clock cycles during runtime. Constant folding only occurs for strings having a length of less than 21.
(Why? Imagine the size of .pyc file generated as a result of the expression 'a'*10**10).
Here's the implementation source for the same.
Note: In Python 3.7, Constant folding was moved out from peephole optimizer to the new AST optimizer
with some change in logic as well, so the fourth snippet doesn't work for Python 3.7.
You can read more about the change here.
▶ Be careful with chained operations
>>> (False == False) in [False] # makes sense
False
>>> False == (False in [False]) # makes sense
False
>>> False == False in [False] # now what?
True
>>> True is False == False
False
>>> False is False is False
True
>>> 1 > 0 < 1
True
>>> (1 > 0) < 1
False
>>> 1 > (0 < 1)
False
Formally, if a, b, c, ..., y, z are expressions and op1, op2, ..., opN are comparison operators,
then a op1 b op2 c ... y opN z is equivalent to a op1 b and b op2 c and ... y opN z,
except that each expression is evaluated at most once.
While such behavior might seem silly to you in the above examples,
it's fantastic with stuff like a == b == c and 0 <= x <= 100.
False is False is False is equivalent to (False is False) and (False is False)
True is False == False is equivalent to (True is False) and (False == False)
and since the first part of the statement (True is False) evaluates to False, the overall expression evaluates to False.
1 > 0 < 1 is equivalent to (1 > 0) and (0 < 1) which evaluates to True.
The expression (1 > 0) < 1 is equivalent to True < 1 and
>>> int(True)
1
>>> True + 1 # not relevant for this example, but just for fun
2
So, 1 < 1 evaluates to False
▶ How not to use is operator
The following is a very famous example present all over the internet.
1.
>>> a = 256
>>> b = 256
>>> a is b
True
>>> a = 257
>>> b = 257
>>> a is b
False
2.
>>> a = []
>>> b = []
>>> a is b
False
>>> a = tuple()
>>> b = tuple()
>>> a is b
True
3.
Output
>>> a, b = 257, 257
>>> a is b
True
Output (Python 3.7.x specifically)
>>> a, b = 257, 257
>>> a is b
False
💡 Explanation:
The difference between is and ==
is operator checks if both the operands refer to the same object (i.e., it checks if the identity of the operands matches or not).
== operator compares the values of both the operands and checks if they are the same.
So is is for reference equality and == is for value equality. An example to clear things up,
>>> class A: pass
>>> A() is A() # These are two empty objects at two different memory locations.
False
256 is an existing object but 257 isn't
When you start up python the numbers from -5 to 256 will be allocated. These numbers are used a lot, so it makes sense just to have them ready.
The current implementation keeps an array of integer objects for all integers between -5 and 256, when you create an int in that range you just get back a reference to the existing object. So it should be possible to change the value of 1. I suspect the behavior of Python, in this case, is undefined. :-)
>>> id(256)
10922528
>>> a = 256
>>> b = 256
>>> id(a)
10922528
>>> id(b)
10922528
>>> id(257)
140084850247312
>>> x = 257
>>> y = 257
>>> id(x)
140084850247440
>>> id(y)
140084850247344
Here the interpreter isn't smart enough while executing y = 257 to recognize that we've already created an integer of the value 257, and so it goes on to create another object in the memory.
Similar optimization applies to other immutable objects like empty tuples as well. Since lists are mutable, that's why [] is [] will return False and () is () will return True. This explains our second snippet. Let's move on to the third one,
Both a and b refer to the same object when initialized with same value in the same line.
Output
>>> a, b = 257, 257
>>> id(a)
140640774013296
>>> id(b)
140640774013296
>>> a = 257
>>> b = 257
>>> id(a)
14064077401
wtfpython (satwikkansal/wtfpython)
Choose a tag and it will appear here. With no tags selected, all catalog projects are shown.
Searching Database…
Choose a tag and it will appear here. With no tags selected, all catalog projects are shown.
Что это такое?Бесплатная и очень популярная подборка неожиданных, контринтуитивных сниппетов на Python — у каждого показан реальный результат выполнения и объяснение, что происходит под капотом.
Это что-то, что нужно устанавливать, или просто читать?Читать, а не устанавливать — это руководство с кодом и комментариями к нему. Пройти его можно прямо в README, на интерактивном сайте или в виде Jupyter-ноутбука.
Как правильнее с этим работать?Читайте примеры по порядку: сначала угадайте, что выведет сниппет, затем проверьте реальный вывод и прочитайте объяснение. Угадали — идите дальше. Нет — это и есть урок.
Мне нужна какая-то конкретная версия Python, чтобы следить за примерами?Ничего устанавливать не нужно — но если хотите попробовать сниппеты самому, подойдёт любой Python 3. Их проверяли на 3.5.2, и они должны работать во всех версиях, если не сказано иное; для примера с оператором walrus нужен 3.8+.
Сниппеты всегда будут вести себя на моей машине одинаково?Не всегда — некоторые примеры ведут себя по-разному в интерактивной оболочке и в скрипте (интернирование строк), а несколько сниппетов прямо помечены как неактуальные для современных версий Python.
Я пишу на Python уже много лет — мне это ещё полезно?Да — автор прямо приглашает опытных питонистов воспринять это как челлендж и попытаться с первого раза правильно предсказать каждый вывод.
А как выглядит одна из таких ловушек?Классика: a = 256; b = 256 делает a is b истинным, а с 257 — уже ложным: CPython кэширует целые числа от -5 до 256, поэтому 257 оказывается двумя разными объектами.
Как устроен каждый пример?Всегда по одной схеме: подготовительный код, реальный вывод (с указанием версии Python, к которой он относится), необязательная строка о том, в чём подвох, и объяснение по пунктам, почему так происходит.
Годится для подготовки к собеседованиям по Python?Это одно из самых популярных применений — GitHub даже помечает репозиторий тегом python-interview-questions, и многие примеры — классические ловушки с собеседований вроде изменяемых аргументов по умолчанию.
Есть ли версии на других языках?Да — есть переводы от сообщества на китайский, вьетнамский, испанский, корейский, русский, немецкий и персидский.
Бесплатная и очень популярная подборка неожиданных, контринтуитивных сниппетов на Python — у каждого показан реальный результат выполнения и объяснение, что происходит под капотом.