Skip to content

release 0.2.3 #10

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
math3usmartins merged 2 commits into from
Jun 13, 2026
Merged

release 0.2.3 #10

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d2f6e6d
fix(semantic-tokens): classify turbofish type args by resolved kind
math3usmartins Jun 13, 2026
5a3dcaa
ci(release): bump version in server info sent to clients
math3usmartins Jun 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 19 additions & 3 deletions features/understand/semantic_tokens.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Feature: Semantic tokens
When I request "textDocument/semanticTokens/full" for "/closure.xphp"
Then a "typeParameter" token covers "T" in "/closure.xphp"

Scenario: Highlight every type argument of a turbofish call, lowercase scalar included
Scenario: Highlight every type argument of a turbofish call by its resolved kind
Given the file at "/turbofish.xphp" contains the following lines:
"""
<?php
Expand All @@ -36,8 +36,24 @@ Feature: Semantic tokens
"""
And the FQN index has been warmed on initialize
When I request "textDocument/semanticTokens/full" for "/turbofish.xphp"
Then a "typeParameter" token covers "int" in "/turbofish.xphp"
And a "typeParameter" token covers "User" in "/turbofish.xphp"
Then a "type" token covers "int" in "/turbofish.xphp"
And a "class" token covers "User" in "/turbofish.xphp"
And a "operator" token covers "::" in "/turbofish.xphp"
And a "operator" token covers "<" in "/turbofish.xphp"
And a "operator" token covers ">" in "/turbofish.xphp"

Scenario: Forward a type parameter through a turbofish inside a generic body
Given the file at "/forward.xphp" contains the following lines:
"""
<?php
namespace App;
class Box<T> {
public function make(): mixed { return Inner::create::<T>(); }
}
"""
And the FQN index has been warmed on initialize
When I request "textDocument/semanticTokens/full" for "/forward.xphp"
Then a "typeParameter" token covers "T" in "/forward.xphp"

Scenario: Multiline block comment highlights on every physical line
Given the file at "/doc.xphp" contains the following lines:
Expand Down
125 changes: 97 additions & 28 deletions src/Handler/SemanticTokens/AstVisitor.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,14 +95,21 @@ public function visit(array $stmts): array
// half the response size at every parameter.
$specs = [];
$reclassifyVariableAt = [];
// File-wide set of every type-param name declared anywhere in the file
// (`name => true`), accumulated by the AST walk from ATTR_GENERIC_PARAMS /
// ATTR_METHOD_GENERIC_PARAMS. The token pass consults it to tell a
// forwarded in-scope type variable (`Foo::create::<T>()` inside a
// generic body -- `typeParameter`) from a concrete named type argument
// (`Foo::create::<User>()` -- `class`).
$declaredTypeParamNames = [];

if ($stmts !== []) {
$traverser = new NodeTraverser();
$traverser->addVisitor($this->newAstWalker($specs, $reclassifyVariableAt));
$traverser->addVisitor($this->newAstWalker($specs, $reclassifyVariableAt, $declaredTypeParamNames));
$traverser->traverse($stmts);
}

$this->collectFromTokens($specs, $reclassifyVariableAt);
$this->collectFromTokens($specs, $reclassifyVariableAt, $declaredTypeParamNames);

return $specs;
}
Expand All @@ -119,9 +126,13 @@ public function visit(array $stmts): array
* T_VARIABLE starts at that offset
* we emit the alternative type
* INSTEAD of `variable`.
* @param array<string, true> $declaredTypeParamNames set of type-param names declared
* anywhere in the file; used to
* paint a forwarded `T` inside a
* turbofish as `typeParameter`.
* @param list<TokenSpec> $out
*/
private function collectFromTokens(array &$out, array $reclassifyVariableAt = []): void
private function collectFromTokens(array &$out, array $reclassifyVariableAt = [], array $declaredTypeParamNames = []): void
{
// Non-strict tokenization (flags=0). TOKEN_PARSE turns
// PhpToken into a strict-mode tokenizer that throws ParseError
Expand All @@ -139,11 +150,19 @@ private function collectFromTokens(array &$out, array $reclassifyVariableAt = []
// identifier or backslash (FQN start). This rejects
// `$size < count($items)` (LHS is T_VARIABLE, not T_STRING)
// and `Foo::BAR < 5` (RHS is a number, not uppercase ident).
// Inside a clause: T_STRING tokens emit as `typeParameter`,
// backslashes as part of FQNs (left unclassified -- the
// surrounding T_STRING segments paint). Depth-counted so
// nested `Box<Lst<T>>` still classifies T.
$genericDepth = 0;
//
// `$clauseKindStack` is a stack of clause KINDS parallel to the
// clause depth (`count($clauseKindStack)`): `'decl'` for a
// declaration clause (`class Box<T>`, `fn<T>`) and `'call'` for a
// call-site turbofish (`Box::<int>`). The kind decides how an
// in-clause identifier is classified: in a `'decl'` clause the
// names are formal type params (`typeParameter`); in a `'call'`
// clause they are concrete type arguments (`type` for builtins,
// `class` for named user types, `typeParameter` only for a
// forwarded in-scope type var). Nested clauses inherit the
// parent kind so `Box::<Lst<T>>` keeps the inner `Lst<T>` as
// call-site args.
$clauseKindStack = [];
$lastSignificantTokenId = null;

$tokenCount = count($tokens);
Expand All @@ -164,8 +183,12 @@ private function collectFromTokens(array &$out, array $reclassifyVariableAt = []

// Open / close angle-clause state on single-char tokens.
if (!$isNamedToken && $token->text === '<') {
if ($genericDepth > 0) {
$genericDepth++;
$openedKind = null;
if ($clauseKindStack !== []) {
// Nested clause: inherit the enclosing kind so
// `Box::<Lst<T>>` keeps the inner `Lst<T>` as call-site
// args.
$openedKind = end($clauseKindStack);
} elseif ($lastSignificantTokenId === T_STRING
&& self::peekIsUppercaseIdent($tokens, $i + 1)
&& self::nameBeforeAngleIsDeclaration($tokens, $i)
Expand All @@ -177,14 +200,14 @@ private function collectFromTokens(array &$out, array $reclassifyVariableAt = []
// bareword comparison whose left side ends in a name --
// `Foo::CONST < Bar`, `MY_CONST < Other` -- from being
// mistaken for a generic declaration.
$genericDepth = 1;
$openedKind = 'decl';
} elseif (($lastSignificantTokenId === T_FN || $lastSignificantTokenId === T_FUNCTION)
&& self::peekIsUppercaseIdent($tokens, $i + 1)
) {
// Anonymous generic closure / arrow declaration clause:
// `fn<T>(…)`, `function<T>(…)` -- the `<` follows the
// `fn` / `function` keyword (no name between).
$genericDepth = 1;
$openedKind = 'decl';
} elseif ($lastSignificantTokenId === T_DOUBLE_COLON) {
// Call-site turbofish: `Foo::<T>`, `static::<T>`,
// `$obj->m::<T>` -- the `<` follows the `::` of `::<`. A `::`
Expand All @@ -198,10 +221,25 @@ private function collectFromTokens(array &$out, array $reclassifyVariableAt = []
// dropping every arg's token. Opening on the empty
// `Foo::<>` is harmless: the next `>` closes it immediately
// with nothing classified inside.
$genericDepth = 1;
$openedKind = 'call';
// P2: the turbofish `::` is an operator delimiter. Paint it
// here -- only when the `<` actually opens a clause -- so a
// bare `Foo::BAR` (no following `<`) never colors its `::`.
$colonIdx = self::previousSignificant($tokens, $i - 1);
if ($colonIdx >= 0 && $tokens[$colonIdx]->id === T_DOUBLE_COLON) {
$this->emit($out, $tokens[$colonIdx]->pos, strlen($tokens[$colonIdx]->text), 'operator');
}
}
if ($openedKind !== null) {
$clauseKindStack[] = $openedKind;
// P2: the opening `<` delimiter. Only painted when a clause
// was actually pushed, so a comparison `<` stays uncolored.
$this->emit($out, $token->pos, 1, 'operator');
}
} elseif (!$isNamedToken && $token->text === '>' && $genericDepth > 0) {
$genericDepth--;
} elseif (!$isNamedToken && $token->text === '>' && $clauseKindStack !== []) {
array_pop($clauseKindStack);
// P2: the closing `>` delimiter.
$this->emit($out, $token->pos, 1, 'operator');
}

// Classify the token.
Expand All @@ -213,15 +251,37 @@ private function collectFromTokens(array &$out, array $reclassifyVariableAt = []
// (single spec, half the response size).
$type = $reclassifyVariableAt[$token->pos];
}
if ($type === null && $genericDepth > 0 && self::isIdentInGenericClause($token->id)) {
// Inside a generic clause an identifier is a type
// name -- emit as `typeParameter` for the LSP-spec
// standard classification. Covers bare T_STRING
// (`T`) and qualified-name tokens
// (T_NAME_FULLY_QUALIFIED `\Stringable`,
// T_NAME_QUALIFIED `App\Foo`, T_NAME_RELATIVE
// `namespace\Foo`).
$type = 'typeParameter';
if ($type === null && $clauseKindStack !== [] && self::isIdentInGenericClause($token->id)) {
// Inside a generic clause an identifier is a type name.
// How it's classified depends on the clause KIND:
//
// - Declaration clause (`class Box<T>`, `fn<T>`): the
// names are formal type parameters -> `typeParameter`.
// Covers bare `T` and bound refs (`<T: \Stringable>`).
//
// - Call-site turbofish (`Box::<int>`,
// `Util::identity::<Banana>`): the names are concrete
// type ARGUMENTS, not formal params. Per LSP semantics
// `typeParameter` denotes a formal variable, so a
// concrete arg must be tokenized as the kind it
// resolves to:
// * a builtin/scalar (`int`, `string`, `void`, ...)
// -> `type`;
// * a forwarded in-scope type variable (`T` passed
// along inside a generic body) -> `typeParameter`;
// * otherwise a named user type -> `class` (the plugin
// colors class/interface/enum identically, so a flat
// `class` is visually correct without resolving the
// exact kind on every keystroke).
if (end($clauseKindStack) === 'decl') {
$type = 'typeParameter';
} elseif ($token->id === T_STRING && self::isReservedWordIdent($token->text)) {
$type = 'type';
} elseif ($token->id === T_STRING && isset($declaredTypeParamNames[$token->text])) {
$type = 'typeParameter';
} else {
$type = 'class';
}
}
if ($type === null && $token->id === T_STRING && self::isReservedWordIdent($token->text)) {
// PHP tokenizes `null`, `true`, `false`, `void`,
Expand Down Expand Up @@ -388,10 +448,15 @@ private static function previousSignificant(array $tokens, int $from): int
* alternative type
* for the T_VARIABLE
* that starts there.
* @param array<string, true> &$declaredTypeParamNames set of every type-param
* name declared in the file;
* lets the token pass paint a
* forwarded `T` in a turbofish
* as `typeParameter`.
*/
private function newAstWalker(array &$out, array &$reclassifyVariableAt): NodeVisitorAbstract
private function newAstWalker(array &$out, array &$reclassifyVariableAt, array &$declaredTypeParamNames): NodeVisitorAbstract
{
$visitor = new class($out, $reclassifyVariableAt, $this) extends NodeVisitorAbstract {
$visitor = new class($out, $reclassifyVariableAt, $declaredTypeParamNames, $this) extends NodeVisitorAbstract {
/**
* Stack of in-scope type-param name sets. Each frame is the
* set of names declared on an enclosing ClassLike via
Expand All @@ -403,12 +468,14 @@ private function newAstWalker(array &$out, array &$reclassifyVariableAt): NodeVi
private array $typeParamStack = [];

/**
* @param list<TokenSpec> $out
* @param array<int, string> $reclassifyVariableAt
* @param list<TokenSpec> $out
* @param array<int, string> $reclassifyVariableAt
* @param array<string, true> $declaredTypeParamNames
*/
public function __construct(
private array &$out,
private array &$reclassifyVariableAt,
private array &$declaredTypeParamNames,
private AstVisitor $emitter,
) {
}
Expand All @@ -422,6 +489,7 @@ public function enterNode(Node $node)
foreach ($params as $param) {
if ($param instanceof \XPHP\Transpiler\Monomorphize\TypeParam) {
$frame[$param->name] = true;
$this->declaredTypeParamNames[$param->name] = true;
}
}
$this->typeParamStack[] = $frame;
Expand Down Expand Up @@ -451,6 +519,7 @@ public function enterNode(Node $node)
foreach ($params as $param) {
if ($param instanceof \XPHP\Transpiler\Monomorphize\TypeParam) {
$frame[$param->name] = true;
$this->declaredTypeParamNames[$param->name] = true;
}
}
}
Expand Down
2 changes: 1 addition & 1 deletion src/LspDispatcherFactory.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -376,7 +376,7 @@ public function create(MessageTransmitter $transmitter, InitializeParams $initia
new ErrorHandlingMiddleware($this->logger),
new InitializeMiddleware($handlers, $eventDispatcher, [
'name' => 'xphp-lsp',
'version' => '0.2.2',
'version' => '0.2.3',
]),
new ShutdownMiddleware($eventDispatcher),
new ResponseHandlingMiddleware($responseWatcher),
Expand Down
Loading
Loading