# API reference

## parsel.csstranslator

### *class* parsel.csstranslator.GenericTranslator(\*args: Any, \*\*kwargs: Any)

Bases: `TranslatorMixin`, [`GenericTranslator`](https://cssselect.readthedocs.io/en/latest/index.html#cssselect.GenericTranslator)

### *class* parsel.csstranslator.HTMLTranslator(\*args: Any, \*\*kwargs: Any)

Bases: `TranslatorMixin`, [`HTMLTranslator`](https://cssselect.readthedocs.io/en/latest/index.html#cssselect.HTMLTranslator)

### *class* parsel.csstranslator.TranslatorMixin(\*args: Any, \*\*kwargs: Any)

Bases: `object`

This mixin adds support to CSS pseudo elements via dynamic dispatch.

Currently supported pseudo-elements are `::text` and `::attr(ATTR_NAME)`.

#### css_to_xpath(css: str, prefix: str = 'descendant-or-self::') → str

#### xpath_attr_functional_pseudo_element(xpath: XPathExpr, function: FunctionalPseudoElement) → XPathExpr

Support selecting attribute values using ::attr() pseudo-element

#### xpath_element(selector: Element) → XPathExpr

#### xpath_pseudo_element(xpath: XPathExpr, pseudo_element: FunctionalPseudoElement | str) → XPathExpr

Dispatch method that transforms XPath to support pseudo-element

#### xpath_text_simple_pseudo_element(xpath: XPathExpr) → XPathExpr

Support selecting text nodes using ::text pseudo-element

### *class* parsel.csstranslator.TranslatorProtocol(\*args, \*\*kwargs)

Bases: [`Protocol`](https://docs.python.org/3/library/typing.html#typing.Protocol)

#### css_to_xpath(css: str, prefix: str = ...) → str

#### xpath_element(selector: Element) → XPathExpr

### *class* parsel.csstranslator.XPathExpr(path: str = '', element: str = '\*', condition: str = '', star_prefix: bool = False)

Bases: `XPathExpr`

#### attribute *: str | None* *= None*

#### *classmethod* from_xpath(xpath: XPathExpr, textnode: bool = False, attribute: str | None = None) → Self

#### join(combiner: str, other: XPathExpr, \*args: Any, \*\*kwargs: Any) → Self

#### textnode *: bool* *= False*

### parsel.csstranslator.css2xpath(query: str) → str

Return translated XPath version of a given CSS query

### parsel.csstranslator.is_safe_name(string, pos=0, endpos=9223372036854775807)

Matches zero or more characters at the beginning of the string.

## parsel.selector

XPath and JMESPath selectors based on the lxml and jmespath Python
packages.

### *exception* parsel.selector.AbsoluteXPathWarning

Bases: `UserWarning`

Emitted when an absolute XPath is used on a nested selector.

### *class* parsel.selector.CTGroupValue

Bases: [`TypedDict`](https://docs.python.org/3/library/typing.html#typing.TypedDict)

### *exception* parsel.selector.CannotDropElementWithoutParent

Bases: `CannotRemoveElementWithoutParent`

### *exception* parsel.selector.CannotRemoveElementWithoutParent

Bases: `Exception`

### *exception* parsel.selector.CannotRemoveElementWithoutRoot

Bases: `Exception`

### *exception* parsel.selector.JSONPathError

Bases: `ValueError`

Raised when a JSONPath query is invalid or cannot be evaluated.

> [!WARNING]
> Added in version VERSION.

### *class* parsel.selector.SafeXMLParser(\*args: Any, \*\*kwargs: Any)

Bases: [`XMLParser`](https://lxml.de/apidoc/lxml.etree.html#lxml.etree.XMLParser)

### *class* parsel.selector.Selector(text: str | None = None, type: str | None = None, body: bytes | bytearray = b'', encoding: str = 'utf-8', namespaces: Mapping[str, str] | None = None, root: Any | None = \_NOT_SET, base_url: str | None = None, \_expr: str | None = None, huge_tree: bool = True)

Bases: `object`

Wrapper for input data in HTML, JSON, or XML format, that allows
selecting parts of it using selection expressions.

You can write selection expressions in CSS or XPath for HTML and XML
inputs, or in JMESPath or JSONPath for JSON inputs.

`text` is an `str` object.

`body` is a `bytes` or `bytearray` object. It can be used together with the
`encoding` argument instead of the `text` argument.

`type` defines the selector type. It can be `"html"`, `"json"`,
`"xml"` or `"text"`. If not specified, the input is handled as
`"json"` if it is a JSON object or array, as `"xml"` if it starts with
an XML declaration, and as `"html"` otherwise.

`base_url` allows setting a URL for the document. This is needed when looking up external entities with relative paths.
See the documentation for [`lxml.etree.fromstring()`](https://lxml.de/apidoc/lxml.etree.html#lxml.etree.fromstring) for more information.

`huge_tree` controls the lxml/libxml2 feature that forbids parsing
certain large documents to protect from possible memory exhaustion. The
argument is `True` by default, which disables the protection to allow
parsing such documents. Set it to `False` if you want to enable the
protection. See [this lxml FAQ entry](https://lxml.de/FAQ.html#is-lxml-vulnerable-to-xml-bombs) for more
information.

#### *property* attrib *: dict[str, str]*

Return the attributes dictionary for underlying element.
For JSON selectors, return an empty dict.

#### body

#### css(query: str) → SelectorList[Self]

Apply the given CSS selector and return a `SelectorList` instance.

`query` is a string containing the CSS selector to apply.

In the background, CSS queries are translated into XPath queries using
[cssselect](https://pypi.python.org/pypi/cssselect/) library and run `.xpath()` method.

#### drop() → None

Drop matched nodes from the parent element.

#### extract() → Any

Serialize and return the matched nodes.

For HTML and XML, the result is always a string.

#### get() → Any

Serialize and return the matched nodes.

For HTML and XML, the result is always a string.

#### getall() → list[str]

Serialize and return the matched node in a 1-element list of strings.

#### jmespath(query: str, \*\*kwargs: Any) → SelectorList[Self]

Find objects matching the JMESPath `query` and return the result as a
`SelectorList` instance with all elements flattened. List
elements implement `Selector` interface too.

`query` is a string containing the [JMESPath](https://jmespath.org/) query to apply.

Any additional named arguments are passed to the underlying
`jmespath.search` call, e.g.:

```
selector.jmespath('author.name', options=jmespath.Options(dict_cls=collections.OrderedDict))
```

#### jsonpath(query: str) → SelectorList[Self]

Find objects matching the JSONPath `query` and return the result as a
`SelectorList` instance with all elements flattened. List
elements implement `Selector` interface too.

> [!WARNING]
> Added in version VERSION.

`query` is a string containing the [JSONPath](https://datatracker.ietf.org/doc/html/rfc9535) query to apply.
Invalid queries raise `JSONPathError`.

Requires a JSONPath package, see [JSONPath support](installation.md).

#### namespaces

#### re(regex: str | Pattern[str], replace_entities: bool = True) → list[str]

Apply the given regex and return a list of strings with the
matches.

`regex` can be either a compiled regular expression or a string which
will be compiled to a regular expression using `re.compile(regex)`.

By default, character entity references are replaced by their
corresponding character (except for `&` and `<`).
Passing `replace_entities` as `False` switches off these
replacements.

#### re_first(regex: str | Pattern[str], default: None = None, replace_entities: bool = True) → str | None

#### re_first(regex: str | Pattern[str], default: str, replace_entities: bool = True) → str

Apply the given regex and return the first string which matches. If
there is no match, return the default value (`None` if the argument
is not provided).

By default, character entity references are replaced by their
corresponding character (except for `&` and `<`).
Passing `replace_entities` as `False` switches off these
replacements.

#### register_namespace(prefix: str, uri: str) → None

Register the given namespace to be used in this `Selector`.
Without registering namespaces you can’t select or extract data from
non-standard namespaces. See [Working on XML (and namespaces)](usage.md).

#### remove_namespaces() → None

Remove all namespaces, allowing to traverse the document using
namespace-less xpaths. See [Removing namespaces](usage.md).
For JSON selectors, this method does nothing.

#### root *: Any*

#### selectorlist_cls

alias of `SelectorList`[Selector]

#### type

#### xpath(query: str, namespaces: Mapping[str, str] | None = None, \*\*kwargs: Any) → SelectorList[Self]

Find nodes matching the xpath `query` and return the result as a
`SelectorList` instance with all elements flattened. List
elements implement `Selector` interface too.

`query` is a string containing the XPATH query to apply.

`namespaces` is an optional `prefix: namespace-uri` mapping (dict)
for additional prefixes to those registered with `register_namespace(prefix, uri)`.
Contrary to `register_namespace()`, these prefixes are not
saved for future calls.

Any additional named arguments can be used to pass values for XPath
variables in the XPath expression, e.g.:

```
selector.xpath('//a[href=$url]', url="http://www.example.com")
```

### *class* parsel.selector.SelectorList(iterable=(), /)

Bases: `list`[`_SelectorType`]

The `SelectorList` class is a subclass of the builtin `list`
class, which provides a few additional methods.

#### *property* attrib *: Mapping[str, str]*

Return the attributes dictionary for the first element.
If the list is empty, return an empty dict.

#### css(query: str) → SelectorList[\_SelectorType]

Call the `.css()` method for each element in this list and return
their results flattened as another `SelectorList`.

`query` is the same argument as the one in `Selector.css()`

#### drop() → None

Drop matched nodes from the parent for each element in this list.

#### extract() → list[str]

Call the `.get()` method for each element in this list and return
their results flattened, as a list of strings.

#### extract_first(default: str | None = None) → Any

Return the result of `.get()` for the first element in this list.
If the list is empty, return the default value.

#### get(default: None = None) → str | None

#### get(default: str) → str

Return the result of `.get()` for the first element in this list.
If the list is empty, return the default value.

#### getall() → list[str]

Call the `.get()` method for each element in this list and return
their results flattened, as a list of strings.

#### jmespath(query: str, \*\*kwargs: Any) → SelectorList[\_SelectorType]

Call the `.jmespath()` method for each element in this list and return
their results flattened as another `SelectorList`.

`query` is the same argument as the one in `Selector.jmespath()`.

Any additional named arguments are passed to the underlying
`jmespath.search` call, e.g.:

```
selector.jmespath('author.name', options=jmespath.Options(dict_cls=collections.OrderedDict))
```

#### jsonpath(query: str) → SelectorList[\_SelectorType]

Call the `.jsonpath()` method for each element in this list and return
their results flattened as another `SelectorList`.

> [!WARNING]
> Added in version VERSION.

`query` is the same argument as the one in `Selector.jsonpath()`.

#### re(regex: str | Pattern[str], replace_entities: bool = True) → list[str]

Call the `.re()` method for each element in this list and return
their results flattened, as a list of strings.

By default, character entity references are replaced by their
corresponding character (except for `&` and `<`).
Passing `replace_entities` as `False` switches off these
replacements.

#### re_first(regex: str | Pattern[str], default: None = None, replace_entities: bool = True) → str | None

#### re_first(regex: str | Pattern[str], default: str, replace_entities: bool = True) → str

Call the `.re()` method for elements in this list and return the
first matching result as a string. If the list is empty or the
regex doesn’t match anything, return the default value (`None` if
the argument is not provided).

By default, character entity references are replaced by their
corresponding character (except for `&` and `<`).
Passing `replace_entities` as `False` switches off these
replacements.

#### xpath(xpath: str, namespaces: Mapping[str, str] | None = None, \*\*kwargs: Any) → SelectorList[\_SelectorType]

Call the `.xpath()` method for each element in this list and return
their results flattened as another `SelectorList`.

`xpath` is the same argument as the one in `Selector.xpath()`

`namespaces` is an optional `prefix: namespace-uri` mapping (dict)
for additional prefixes to those registered with `register_namespace(prefix, uri)`.
Contrary to `register_namespace()`, these prefixes are not
saved for future calls.

Any additional named arguments can be used to pass values for XPath
variables in the XPath expression, e.g.:

```
selector.xpath('//a[href=$url]', url="http://www.example.com")
```

### parsel.selector.create_root_node(text: str, parser_cls: type[XMLParser | HTMLParser], base_url: str | None = None, huge_tree: bool = True, body: bytes = b'', encoding: str = 'utf-8') → \_Element

Create root node for text using given parser class.

## parsel.utils

### parsel.utils.extract_regex(regex: str | Pattern[str], text: str, replace_entities: bool = True) → list[str]

Extract a list of strings from the given text using the following policies:
\* if the regex contains a named group called “extract” that will be returned
\* if the regex contains multiple numbered groups, all those will be returned (flattened)
\* if the regex doesn’t contain any group the entire regex matching is returned

### parsel.utils.flatten(sequence) → list

Returns a single, flat list which contains all elements retrieved
from the sequence and all recursively contained sub-sequences
(iterables).
Examples:
>>> flatten([[[1,2,3], (42,None)], [4,5], [6], 7, (8,9,10)])
[1, 2, 3, 42, None, 4, 5, 6, 7, 8, 9, 10]
>>> flatten([“foo”, “bar”])
[‘foo’, ‘bar’]
>>> flatten([“foo”, [“baz”, 42], “bar”])
[‘foo’, ‘baz’, 42, ‘bar’]

### parsel.utils.iflatten(sequence) → Iterator

Similar to `flatten()`, but returns iterator instead
Examples:
>>> list(iflatten([[1, 2], (3, 4)]))
[1, 2, 3, 4]

### parsel.utils.shorten(text: str, width: int, suffix: str = '...') → str

Truncate the given text to fit in the given width.