Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 18 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -250,11 +250,15 @@ With the `@superfunction` decorator, you no longer need to call special methods

If you use it as a regular function, a regular function will be created "under the hood" based on the template and then called:

To call a superfunction like a regular function, you need to use a special tilde syntax:

```python
my_superfunction()
~my_superfunction()
#> so, it's just usual function!
```

Yes, the tilde syntax simply means putting the `~` symbol in front of the function name when calling it.

If you use `asyncio.run` or the `await` keyword when calling, the async version of the function will be automatically generated and called:

```python
Expand All @@ -273,15 +277,22 @@ list(my_superfunction())

How does it work? In fact, `my_superfunction` returns some kind of intermediate object that can be both a coroutine and a generator and an ordinary function. Depending on how it is handled, it lazily code-generates the desired version of the function from a given template and uses it.

Separately, it is worth considering how the superfunction works in the normal function mode. The point is that we need to somehow distinguish a call wrapped with an `await` statement or iteration from a call in which we use a function as a regular function. To do this, a special trick is used by default: assigning a finalizer to reset the reference counter to a variable. When the reference count is zero, the normal (synchronous) implementation of the function is automatically called. However, this imposes 2 restrictions:
By default, a superfunction is called as a regular function using tilde syntax, but there is another mode. To enable it, use the appropriate flag in the decorator:

- You cannot use the return values from this function in any way. This works in the coroutine function mode, but not in the regular mode. If you try to save the result of a function call to a variable, the reference counter to the returned object will not reset while this variable exists, and accordingly the function will not actually be called.
- Exceptions will not work normally inside this function. Rather, they can be picked up and intercepted in [`sys.unraisablehook`](https://docs.python.org/3/library/sys.html#sys.unraisablehook), but they will not go up the stack above this function. This is due to a feature of CPython: exceptions that occur inside callbacks for finalizing objects are completely escaped.
```python
@superfunction(tilde_syntax=False)
```

To get around both of these problems, you can use a special syntactic trick: put the `~` symbol before calling the function. Like this:
In this case, the superfunction can be called in exactly the same way as a regular function:

```python
~my_superfunction()
my_superfunction()
#> so, it's just usual function!
```

In this case, the behavior of the superfunction will be completely indistinguishable from the behavior of a regular function. Return expressions and exceptions will work exactly as you expect them to.
However, it is not completely free. The fact is that this mode uses a special trick with a reference counter, a special mechanism inside the interpreter that cleans up memory. When there is no reference to an object, the interpreter deletes it, and you can link your callback to this process. It is inside such a callback that the contents of your function are actually executed. This imposes some restrictions on you:

- You cannot use the return values from this function in any way. If you try to save the result of a function call to a variable, the reference counter to the returned object will not reset while this variable exists, and accordingly the function will not actually be called.
- Exceptions will not work normally inside this function. Rather, they can be picked up and intercepted in [`sys.unraisablehook`](https://docs.python.org/3/library/sys.html#sys.unraisablehook), but they will not go up the stack above this function. This is due to a feature of CPython: exceptions that occur inside callbacks for finalizing objects are completely escaped.

This mode is well suited for functions such as logging or sending statistics from your code: simple functions from which no exceptions or return values are expected. In all other cases, I recommend using the tilde syntax.
4 changes: 3 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "transfunctions"
version = "0.0.4"
version = "0.0.5"
authors = [
{ name="Evgeniy Blinov", email="zheni-b@yandex.ru" },
]
Expand Down Expand Up @@ -32,6 +32,7 @@ classifiers = [
'License :: OSI Approved :: MIT License',
'Intended Audience :: Developers',
'Topic :: Software Development :: Libraries',
'Topic :: Software Development :: Code Generators',
'Framework :: AsyncIO',
]
keywords = [
Expand All @@ -40,6 +41,7 @@ keywords = [
'async to sync',
'code generation',
'ast manipulation',
'metaprogramming',
'magic',
]

Expand Down
101 changes: 99 additions & 2 deletions tests/units/decorators/test_superfunction.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import io
import sys
from asyncio import run
from contextlib import redirect_stdout

Expand All @@ -22,7 +23,7 @@
"""


def test_just_sync_call():
def test_just_sync_call_without_breackets():
@superfunction
def function():
with sync_context:
Expand All @@ -32,12 +33,44 @@ def function():
with generator_context:
yield from [1, 2, 3]

buffer = io.StringIO()
with redirect_stdout(buffer):
~function()
assert buffer.getvalue() == "1\n"


def test_just_sync_call_without_tilde_syntax():
@superfunction(tilde_syntax=False)
def function():
with sync_context:
print(1)
with async_context:
print(2)
with generator_context:
yield from [1, 2, 3]

buffer = io.StringIO()
with redirect_stdout(buffer):
function()
assert buffer.getvalue() == "1\n"


def test_just_sync_call_with_tilde_syntax():
@superfunction(tilde_syntax=True)
def function():
with sync_context:
print(1)
with async_context:
print(2)
with generator_context:
yield from [1, 2, 3]

buffer = io.StringIO()
with redirect_stdout(buffer):
~function()
assert buffer.getvalue() == "1\n"


def test_just_async_call():
@superfunction
def function():
Expand Down Expand Up @@ -84,7 +117,7 @@ def function(a, b):

buffer = io.StringIO()
with redirect_stdout(buffer):
function(1, 2)
~function(1, 2)
assert buffer.getvalue() == "1\n"


Expand Down Expand Up @@ -248,3 +281,67 @@ def template():

with pytest.raises(WrongDecoratorSyntaxError, match=full_match('The @superfunction decorator cannot be used in conjunction with other decorators.')):
~template()


def test_pass_coroutine_function_to_decorator():
with pytest.raises(ValueError, match=full_match("Only regular or generator functions can be used as a template for @superfunction. You can't use async functions.")):
@superfunction
async def function_maker():
return 4


def test_pass_not_function_to_decorator():
with pytest.raises(ValueError, match=full_match("Only regular or generator functions can be used as a template for @superfunction.")):
superfunction(1)


def test_try_to_pass_lambda_to_decorator():
with pytest.raises(ValueError, match=full_match("Only regular or generator functions can be used as a template for @superfunction. Don't use lambdas here.")):
superfunction(lambda x: x)


def test_choose_tilde_syntax_off_and_use_tilde():
@superfunction(tilde_syntax=False)
def function():
pass

with pytest.raises(NotImplementedError, match=full_match('The syntax with ~ is disabled for this superfunction. Call it with simple breackets.')):
~function()


def test_call_superfunction_without_tilde_syntax_whet_it_is_on_by_default():
exception_message = None
def temporary_hook(unraisable):
nonlocal exception_message
exception_message = str(unraisable.exc_value)
old_hook = sys.unraisablehook
sys.unraisablehook = temporary_hook

@superfunction
def function():
pass

function()

assert 'The tilde-syntax is enabled for the "function" function. Call it like this: ~function().' == exception_message

sys.unraisablehook = old_hook


def test_call_superfunction_without_tilde_syntax_whet_it_is_on():
exception_message = None
def temporary_hook(unraisable):
nonlocal exception_message
exception_message = str(unraisable.exc_value)
old_hook = sys.unraisablehook
sys.unraisablehook = temporary_hook

@superfunction(tilde_syntax=True)
def function():
pass

function()

assert 'The tilde-syntax is enabled for the "function" function. Call it like this: ~function().' == exception_message

sys.unraisablehook = old_hook
65 changes: 39 additions & 26 deletions transfunctions/decorators/superfunction.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
from ast import NodeTransformer, Return, AST
from inspect import currentframe
from functools import wraps
from typing import Dict, Any, Optional, Union, List
from typing import Dict, Any, Optional, Union, List, Callable
from collections.abc import Coroutine

if sys.version_info <= (3, 10): # pragma: no cover
Expand All @@ -23,13 +23,14 @@
CoroutineClass: TypeAlias = Coroutine[Any, Any, None]

class UsageTracer(CoroutineClass):
def __init__(self, args, kwargs, transformer) -> None:
def __init__(self, args, kwargs, transformer, tilde_syntax: bool) -> None:
self.flags: Dict[str, bool] = {}
self.args = args
self.kwargs = kwargs
self.transformer = transformer
self.coroutine = self.async_sleep_option(self.flags, args, kwargs, transformer)
self.finalizer = weakref.finalize(self, self.sync_sleep_option, self.flags, args, kwargs, transformer, self.coroutine)
self.tilde_syntax = tilde_syntax
self.coroutine = self.async_option(self.flags, args, kwargs, transformer)
self.finalizer = weakref.finalize(self, self.sync_option, self.flags, args, kwargs, transformer, self.coroutine, tilde_syntax)

def __iter__(self):
self.flags['used'] = True
Expand All @@ -42,8 +43,12 @@ def __await__(self) -> Any: # pragma: no cover
return self.coroutine.__await__()

def __invert__(self):
result = self.finalizer()
return result
if not self.tilde_syntax:
raise NotImplementedError('The syntax with ~ is disabled for this superfunction. Call it with simple breackets.')

self.flags['used'] = True
self.coroutine.close()
return self.transformer.get_usual_function()(*(self.args), **(self.kwargs))

def send(self, value: Any) -> Any:
return self.coroutine.send(value)
Expand All @@ -55,37 +60,45 @@ def close(self) -> None: # pragma: no cover
pass

@staticmethod
def sync_sleep_option(flags: Dict[str, bool], args, kwargs, transformer, wrapped_coroutine: CoroutineClass) -> None:
def sync_option(flags: Dict[str, bool], args, kwargs, transformer, wrapped_coroutine: CoroutineClass, tilde_syntax: bool) -> None:
if not flags.get('used', False):
wrapped_coroutine.close()
return transformer.get_usual_function()(*args, **kwargs)
if not tilde_syntax:
return transformer.get_usual_function()(*args, **kwargs)
else:
raise NotImplementedError(f'The tilde-syntax is enabled for the "{transformer.function.__name__}" function. Call it like this: ~{transformer.function.__name__}().')

@staticmethod
async def async_sleep_option(flags: Dict[str, bool], args, kwargs, transformer) -> None:
async def async_option(flags: Dict[str, bool], args, kwargs, transformer) -> None:
flags['used'] = True
return await transformer.get_async_function()(*args, **kwargs)


not_display(UsageTracer)

def superfunction(function):
class NoReturns(NodeTransformer):
def visit_Return(self, node: Return) -> Optional[Union[AST, List[AST]]]:
raise WrongTransfunctionSyntaxError('A superfunction cannot contain a return statement.')
def superfunction(*args: Callable, tilde_syntax: bool = True):
def decorator(function):
class NoReturns(NodeTransformer):
def visit_Return(self, node: Return) -> Optional[Union[AST, List[AST]]]:
raise WrongTransfunctionSyntaxError('A superfunction cannot contain a return statement.')

transformer = FunctionTransformer(
function,
currentframe().f_back.f_lineno,
'superfunction',
extra_transformers=[
#NoReturns(),
],
)

transformer = FunctionTransformer(
function,
currentframe().f_back.f_lineno,
'superfunction',
extra_transformers=[
#NoReturns(),
],
)
@wraps(function)
def wrapper(*args, **kwargs):
return UsageTracer(args, kwargs, transformer, tilde_syntax)

@wraps(function)
def wrapper(*args, **kwargs):
return UsageTracer(args, kwargs, transformer)
wrapper.__is_superfunction__ = True

wrapper.__is_superfunction__ = True
return wrapper

return wrapper
if args:
return decorator(args[0])
return decorator
2 changes: 2 additions & 0 deletions transfunctions/transformer.py
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,8 @@ def visit_FunctionDef(self, node: FunctionDef) -> Optional[Union[AST, List[AST]]
raise WrongDecoratorSyntaxError(f"The @{decorator_name} decorator can only be used with the '@' symbol. Don't use it as a regular function. Also, don't rename it.")

for decorator in node.decorator_list:
if isinstance(decorator, Call):
decorator = decorator.func
if decorator.id != decorator_name:
raise WrongDecoratorSyntaxError(f'The @{decorator_name} decorator cannot be used in conjunction with other decorators.')
else:
Expand Down
Loading