BUG: UndefinedVariableError in DataFrame.query if name is type

This issue has been created since 2022-09-21.

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

df = pd.DataFrame({'x': [1, 2, 3], 'y': [4, 5, 6]})

class A:
    a = 1

df.query('x == @A.a')  # <-- this produces the error

Issue Description

In Pandas 1.5.0 it is not possible to use class as name in DataFrame.query:

>>> import pandas as pd
>>> 
>>> df = pd.DataFrame({'x': [1, 2, 3], 'y': [4, 5, 6]})
>>> 
>>> class A:
...     a = 1
... 
>>> df.query('x == @A.a')  # <-- this produces the error
Traceback (most recent call last):
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/scope.py", line 198, in resolve
    return self.resolvers[key]
  File "/usr/lib64/python3.10/collections/__init__.py", line 986, in __getitem__
    return self.__missing__(key)            # support subclasses that define __missing__
  File "/usr/lib64/python3.10/collections/__init__.py", line 978, in __missing__
    raise KeyError(key)
KeyError: 'A'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/scope.py", line 209, in resolve
    return self.temps[key]
KeyError: 'A'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/util/_decorators.py", line 317, in wrapper
    return func(*args, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/frame.py", line 4464, in query
    res = self.eval(expr, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/util/_decorators.py", line 317, in wrapper
    return func(*args, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/frame.py", line 4602, in eval
    return _eval(expr, inplace=inplace, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/eval.py", line 354, in eval
    parsed_expr = Expr(expr, engine=engine, parser=parser, env=env)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 813, in __init__
    self.terms = self.parse()
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 832, in parse
    return self._visitor.visit(self.expr)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 415, in visit
    return visitor(node, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 421, in visit_Module
    return self.visit(expr, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 415, in visit
    return visitor(node, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 424, in visit_Expr
    return self.visit(node.value, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 415, in visit
    return visitor(node, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 723, in visit_Compare
    return self.visit(binop)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 415, in visit
    return visitor(node, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 536, in visit_BinOp
    op, op_class, left, right = self._maybe_transform_eq_ne(node)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 458, in _maybe_transform_eq_ne
    right = self.visit(node.right, side="right")
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 415, in visit
    return visitor(node, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 645, in visit_Attribute
    resolved = self.visit(value).value
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 415, in visit
    return visitor(node, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/expr.py", line 549, in visit_Name
    return self.term_type(node.id, self.env, **kwargs)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/ops.py", line 85, in __init__
    self._value = self._resolve_name()
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/ops.py", line 109, in _resolve_name
    res = self.env.resolve(local_name, is_local=is_local)
  File "/home/petr/.local/venv/lib64/python3.10/site-packages/pandas/core/computation/scope.py", line 211, in resolve
    raise UndefinedVariableError(key, is_local) from err
pandas.errors.UndefinedVariableError: name 'A' is not defined

I think it is after this commit:
8b36142

I can get expected behavior if I comment this condition:
https://github.com/pandas-dev/pandas/blob/main/pandas/core/computation/ops.py#L105

Expected Behavior

>>> import pandas as pd
>>> 
>>> df = pd.DataFrame({'x': [1, 2, 3], 'y': [4, 5, 6]})
>>> 
>>> class A:
...     a = 1
... 
>>> df.query('x == @A.a')  # <-- this produces the error
   x  y
0  1  4

Installed Versions

INSTALLED VERSIONS

commit : 87cfe4e
python : 3.10.6.final.0
python-bits : 64
OS : Linux
OS-release : 5.19.9-200.fc36.x86_64
Version : #1 SMP PREEMPT_DYNAMIC Thu Sep 15 09:49:52 UTC 2022
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.5.0
numpy : 1.23.3
pytz : 2022.2.1
dateutil : 2.8.2
setuptools : 65.3.0
pip : 22.2.2
Cython : None
pytest : 7.1.3
hypothesis : None
sphinx : 5.1.1
blosc : None
feather : None
xlsxwriter : 3.0.3
lxml.etree : 4.9.1
html5lib : 1.1
pymysql : None
psycopg2 : 2.9.3
jinja2 : 3.1.2
IPython : 8.5.0
pandas_datareader: None
bs4 : 4.11.1
bottleneck : 1.3.5
brotli : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : 3.6.0
numba : None
numexpr : 2.8.3
odfpy : None
openpyxl : 3.0.10
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : 1.9.1
snappy : None
sqlalchemy : 1.4.41
tables : None
tabulate : 0.8.10
xarray : None
xlrd : 2.0.1
xlwt : None
zstandard : None
tzdata : None

phofl wrote this answer on 2022-09-21

Can confirm

commit 8b36142a0e7c1071e9fd9ff37f579ebdef056838
Author: Khor Chean Wei <[email protected]>
Date:   Fri Aug 12 08:15:48 2022 +0800

    Error raised message: DataFrame eval will not work with 'Timestamp' column name (#47273)

cc @weikhor

rhshadrach wrote this answer on 2022-09-21

Somewhat tangential, but with f-strings, is there any advantage of using @? E.g. in this case

df.query(f'x == {A.a}')

This has the added benefit of linters detecting the usage of A.a.

More Details About Repo
Owner Name pandas-dev
Repo Name pandas
Full Name pandas-dev/pandas
Language Python
Created Date 2010-08-24
Updated Date 2022-09-29
Star Count 35373
Watcher Count 1123
Fork Count 15031
Issue Count 3581

YOU MAY BE INTERESTED

Issue Title Created Date Comment Count Updated Date
Using the gVisor runtime? 0 2021-02-01 2022-05-12
Cannot resolve host darch-demo for test drive image 1 2020-11-22 2022-05-12
Association API implementation does not honor remote AE's maximum PDU length 2 2021-08-31 2022-07-23
Import and Runtime errors with numpy and missing 'service_identity' module 8 2021-07-17 2022-08-27
ldeinit will take segmentation fault if given keyboard/mouse input 5 2022-06-08 2022-09-23
[2021-12-28] New commits in sveltejs/svelte 0 2021-12-28 2022-08-16
This is more like a discussion post than an issue, but you don't have discussions enabled. 2 2021-01-09 2022-09-18
[Google Meet] Create new meet for other than default users e.g. /u/1 1 2022-09-19 2022-09-28
Key-based item uniqueness 44 2018-01-23 2022-09-14
Also have 'longopts' for Lay_L 2 2022-03-18 2022-09-06
[Cascader]级联组件当数据量过大时造成卡顿 3 2022-05-17 2022-09-14
Cannot use `graphics-document` role with `aria-roledescription`. 1 2021-09-28 2021-12-17
[QUESTION]: Can I load CCtextures twice? Alternatively, can I filter among preloaded CCtextures? 5 2022-04-20 2022-09-26
除了群晖能做其他的支持么 ? 比如威联通 5 2021-09-18 2022-09-26
Edit with shareX add to win11 context menu 9 2021-10-07 2022-07-25
Trying to get in touch regarding a security issue 0 2021-09-02 2022-09-26
Music Control, LED Control 0 2021-09-03 2022-08-29
page 573, Figure 17.8 1 2022-04-28 2022-09-12
Call plugin from command line 14 2022-04-07 2022-09-14
XVDPU-TRD C64B5 timeout on VCK190-ES 4 2022-02-08 2022-09-24
Does the color '#c411a6' break randomColor? 3 2020-12-22 2022-09-17
ocamlopt does not recognize `-cmi-file` 1 2022-05-25 2022-09-21
With Docker passwd is not working 1 2022-05-09 2022-09-27
gltfpack: -kn and -mi conflict 2 2021-06-02 2022-07-07
EventEmitter warnings in RN 0.65 2 2021-09-15 2022-08-26
📑 CITATION files 1 2021-07-24 2022-01-13
[Feature] Podcast Support 4 2021-10-08 2022-01-28
Multi line env vars continued 1 2021-06-15 2022-09-10
SAML authenticated - no role in administration 5 2020-10-04 2022-09-27
[BSI][20210621] Usage of Outdated Components 1 2021-06-22 2022-09-24
[scroll-animations-1] Entry/Exit Transitions for View Timeline effects 31 2022-02-14 2022-09-04
bug: SearchTree 搜索徐汇 徐汇里的没有展示 1 2022-03-03 2022-09-23
Changing the entryNames placeholder value generates different (inconsistent) hashes. 2 2021-12-16 2022-09-29
Class ref: Translations hide up-to-date information 4 2021-07-19 2022-08-25
Keep record before start 2 2021-03-05 2022-08-28
AttributeError: 'Job' object has no attribute 'until' 1 2021-04-11 2022-09-26
Allow streaming to any channel 0 2017-07-07 2022-09-26
Boost dependency can be (mostly) upgraded 1 2022-03-30 2022-09-14
[CTEST] "--weight" flag won't pass the expected value to weight tensor 4 2022-03-31 2022-09-20
Using Vim/Neovim as pager 2 2021-01-01 2022-09-12
Too many arguments / Invalid self argument when referencing callable attribute of an object. 3 2021-01-26 2022-09-26
Hashcat is not using full VRAM 1 2022-07-12 2022-09-22
Changing the sort should reset the page number 2 2021-06-28 2022-09-26
Request: stop_circle_filled and stop_circle_outline 1 2021-03-21 2022-09-18
[@typescript-eslint/no-unused-vars-experimental] Got a false positive 1 2021-10-18 2021-12-11
False positives Due to query inability to check Dependency Presence 3 2022-02-03 2022-08-30
Replace deprecated imp 0 2022-06-20 2022-09-17
[Feat]: Textfield: add support for setSelectionRange 0 2022-01-21 2022-09-08
java.lang.AssertionError: illegal type variable reference only happend on Android N or higher versions 5 2017-06-08 2022-09-28
android scenario_app tests documentation / scripts is out of date 11 2021-09-18 2022-09-25