Json Exporter (#9) by ArchooD2 · Pull Request #36 · MuddyHope/pyquerytracker

ArchooD2 · 2025-06-17T00:44:41Z

Should be right, cherrypicked as I was expected to, I hope. (#9)

Copilot

Pull Request Overview

This PR introduces a JSON exporter for query tracking logs along with associated tests and configuration updates. Key changes include:

Integration of a new JsonExporter class in pyquerytracker for threadsafe JSON log exporting.
Addition of tests in tests/exporter/test_json_exporter.py to validate the JSON export functionality.
Updates in core and configuration files to support JSON export and provide a usage example.

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
tests/test_core.py	Duplicate import statements need cleanup.
tests/exporter/test_json_exporter.py	New test coverage for JSON exporting functionality.
pyquerytracker/exporter/json_exporter.py	New exporter class for JSON logs.
pyquerytracker/exporter/init.py	Updated export package initialization.
pyquerytracker/core.py	Integrated JsonExporter but missing log_data initialization.
pyquerytracker/config.py	Updated configuration docstrings for export settings.
examples/exporter/json_exporter.py	Usage example for JSON exporter functionality.

Comments suppressed due to low confidence (2)

tests/test_core.py:5

Duplicate import of 'pytest' detected; consider removing the redundant import to maintain cleaner code.

import pytest

pyquerytracker/core.py:79

The variable 'log_data' is referenced in the successful execution branch but has not been initialized; consider defining 'log_data' with the appropriate log details before its use.

extra=log_data,

ArchooD2 · 2025-06-17T00:45:56Z

oh god what did i break now

ArchooD2 · 2025-06-17T00:47:45Z

MuddyHope please help, I thought I did it the right way

MuddyHope · 2025-06-17T00:51:44Z

Did you cherry pick my commit also? Also where they in the same order??

…

On Mon, Jun 16, 2025 at 5:48 PM Paper ***@***.***> wrote: *ArchooD2* left a comment (MuddyHope/pyquerytracker#36) <#36 (comment)> MuddyHope please help, I thought I did it the right way — Reply to this email directly, view it on GitHub <#36 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOKTJ263APJ5VWKDBH7GE4L3D5QUPAVCNFSM6AAAAAB7OXEIOOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSNZYGU4DAMRTGQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

ArchooD2 · 2025-06-17T00:53:12Z

I believe so for both cases. I can try again, but I think I did it right. 1 moment

ArchooD2 · 2025-06-17T01:00:46Z

refixing this and then reopening.

…for JSON log export

Moved json export testing to new file Changed get_config patch in test to configure() call No longer writes to os.path.dirname(None) (disables JsonExporter class entirely if None path supplied) File writes should now be thread-safe Changed export logic to use pathlib rather than os

…ort (requested changes)

ArchooD2 · 2025-06-17T01:04:08Z

Okay, should be fixed.

MuddyHope · 2025-06-17T01:07:12Z

Thanks I’ll test it out. Have you considered the output to be in json format? (Not just in pytest) Try running the example/exporter file.

…

On Mon, Jun 16, 2025 at 6:04 PM Paper ***@***.***> wrote: Reopened #36 <#36>. — Reply to this email directly, view it on GitHub <#36 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOKTJ26W7O7IH4CF6JMY7B33D5SR5AVCNFSM6AAAAAB7OXEIOOVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJYGE4DANRZHA4TGMY> . You are receiving this because you commented.Message ID: ***@***.***>

ArchooD2 · 2025-06-17T01:22:41Z

Found a minor bug with the json formatting

MuddyHope · 2025-06-17T01:23:50Z

        with _lock, self._path.open("a", encoding="utf-8") as f:
            json.dump(record, f)
-            f.write("\n")
+            f.write(",\n")


The opening and closing brackets ?? []??

resolved in newest push

…log export. Holy hell, making this use json instead of jsonl was tougher than expected.

ArchooD2 · 2025-06-17T01:43:01Z

This is embarassing...

Edit: ARE YOU KIDDING ME LINT

…s, and reintroduce professionalism.

…er class

MuddyHope · 2025-06-17T03:12:11Z

Hi @ArchooD2, can you run the examples/exporter/json_exporter.py function once, the slow_function and faulty_function aren't appearing on the json file. Can you add these also in the test module?

ArchooD2 · 2025-06-17T04:11:37Z

I'm gonna have words to say with an inanimate github workflow (just the linter one, I'm mildly displeased with how often I'm forgetting to lint)

ArchooD2 · 2025-06-17T04:50:51Z

Seems to be working fine now @MuddyHope

MuddyHope · 2025-06-17T04:56:10Z

Hi @ArchooD2, the previous commits were good, you changed after my last review. Also, the flush wasn't so necessary. If you can remove it, that would be good. When I ran the examples/exporter files twice, there was no appending. Which should be necessary to save these queries.

Take your time and test it once, with the existing features and submit the PR. Hoping to see these changes.

…ssary instantiation

…es. Added sanity checks in test_json_exporter for use in the event of further bugs.

https://stackoverflow.com/questions/73568036/flake8-line-break-before-binary-operator-how-to-fix-it I may suggest a change to "extend-ignore," but for now I've simply remedied the problem at hand.

MuddyHope

If you are using AI to write your code, it's better to give it more context; try using Cursor.

MuddyHope · 2025-06-17T04:45:19Z

+def flush_exported_logs() -> None:
+    """
+    Manually flushes the JSON exporter buffer to disk.
+    Useful for ensuring log data is written in real-time or during tests.


Can all these exporter functions be put in utils?

MuddyHope · 2025-06-17T04:46:27Z

@@ -1,3 +1,5 @@
+# pyquerytracker/config.py


Can you remove this line?

MuddyHope · 2025-06-17T04:48:55Z

 import logging
 import pytest
 from pyquerytracker import TrackQuery
+import pytest


Why two imports here?

Fixed, accidentally added. ✔️

MuddyHope · 2025-06-17T04:51:41Z

                        self.config.slow_log_level,
-                        f"{class_name}.{func.__name__} -> Slow execution: took %.2fms",
+                        "%s.%s → Slow execution: took %.2fms",
+                        class_name or "<module>",


what is this ?

MuddyHope · 2025-06-17T21:12:13Z

+class _ExporterManager:
+    _instance: JsonExporter | None = None
+
+    @classmethod


Check if it's really necessary, if it's really necessary, keep it in the utils file.

…tions to pyquerytracker/exporter/utils.py for better structure

…p whitespace in utils.py

MuddyHope · 2025-06-18T01:43:36Z

Usemake lint to refractor and checking linting issues.

ArchooD2 · 2025-06-18T02:13:10Z

make lint? 1 sec.

ArchooD2 · 2025-06-18T02:46:53Z

Realized way too late that my choco wasn't installed properly, so make just didn't work...

MuddyHope · 2025-06-18T04:48:39Z

Are you done? @ArchooD2 ?
If not, can you remove the flush, we don't want to delete the existing one, it should append to the file.
Right now it is flushing the data.

… method

ArchooD2 · 2025-06-18T06:21:20Z

Updated the flush logic so that it appends! I should note that examples/exporter/json_exporter.py is much faster

ArchooD2 · 2025-06-19T01:12:17Z

I think I'm done, if there aren't any more problems.

MuddyHope · 2025-06-19T05:40:02Z

Hi @ArchooD2, there are some conflicts. There was a change made to the framework of the exporter class.

ArchooD2 · 2025-06-19T14:11:53Z

I'm sorry, but how many changes to main have there been while I was working on this?

FOLLOWUP, WHY DID IT NEVER TELL ME

…??????????????????????????????????????????????????????????????????????????????? (Fix pytest fail and linting issues.)

MuddyHope · 2025-06-19T19:29:25Z

Hi @ArchooD2, can you make the changes in accordance with the framework. It should be easier now with having a JsonExporter as there is already a BaseClass that have you to inherit from.

MuddyHope · 2025-06-22T20:31:41Z

@ArchooD2, are you working on this?

ArchooD2 · 2025-06-22T22:45:27Z

I think I need to take some time away for a little, please allow anyone else to make a json exporter for this project 👍

Copilot AI review requested due to automatic review settings June 17, 2025 00:44

Copilot AI reviewed Jun 17, 2025

View reviewed changes

ArchooD2 mentioned this pull request Jun 17, 2025

2nd attempt at adding json exporting! #24

Closed

ArchooD2 closed this Jun 17, 2025

ArchooD2 force-pushed the cherry-picked-json-exporter branch from bf2d19c to c3d2ff3 Compare June 17, 2025 01:00

PJSans and others added 7 commits June 16, 2025 19:01

feat: add export functionality for query tracking logs

88c53aa

lint: black . command ran

a4d9605

docs: add export_log method docstring to clarify parameters (pylint...)

1a74cf3

fix: clean up docstring formatting in export_log method and add test …

775109b

…for JSON log export

Examples test runs for Export added

2dda6a6

fix: update log file extension to .json and add debug logging for exp…

4a743c5

…ort (requested changes)

ArchooD2 reopened this Jun 17, 2025

fix: change JSON export format to append a comma after each record

ea45661

MuddyHope requested changes Jun 17, 2025

View reviewed changes

feat: implement manual flushing of JSON exporter and update test for …

e757db0

…log export. Holy hell, making this use json instead of jsonl was tougher than expected.

PJSans added 3 commits June 16, 2025 19:45

refactor: implement _ExporterManager to shut pylint up

7ecab13

refactor: Rewrite comment to clarify purpose of _ExporterManager clas…

4f4feb2

…s, and reintroduce professionalism.

fix: add space after comment for better readability in _ExporterManag…

5e51a84

…er class

ArchooD2 requested a review from MuddyHope June 17, 2025 01:51

MuddyHope linked an issue Jun 17, 2025 that may be closed by this pull request

Add JSON Exporter #9

Closed

ArchooD2 force-pushed the cherry-picked-json-exporter branch from a331b2c to 5e51a84 Compare June 17, 2025 06:15

PJSans added 3 commits June 17, 2025 00:20

refactor: delay exporter initialization in TrackQuery to avoid unnece…

b8fcb38

…ssary instantiation

refactor: Add functionality to auto-update exporter when config chang…

7cbaa42

…es. Added sanity checks in test_json_exporter for use in the event of further bugs.

fix: update flake8 ignore rules to include W503 and W504

9fe8183

https://stackoverflow.com/questions/73568036/flake8-line-break-before-binary-operator-how-to-fix-it I may suggest a change to "extend-ignore," but for now I've simply remedied the problem at hand.

MuddyHope requested changes Jun 18, 2025

View reviewed changes

PJSans added 2 commits June 17, 2025 18:40

refactor: reorganize imports and move _ExporterManager class and func…

34bfd31

…tions to pyquerytracker/exporter/utils.py for better structure

refactor: remove unused import from test_json_exporter.py and clean u…

2e0219e

…p whitespace in utils.py

refactor: simplify log message in TrackQuery for clarity

b667b7c

PJSans added 2 commits June 18, 2025 00:15

fix: change file open mode from write to append in JsonExporter flush…

a01218f

… method

change: Update exporter to append to files instead of rewriting.

a8a8c32

ArchooD2 requested a review from MuddyHope June 19, 2025 01:11

ArchooD2 and others added 2 commits June 19, 2025 08:16

Merge branch 'main' into cherry-picked-json-exporter

a1a028b

?????????????????????????????????????????????????????????????????????…

4133594

…??????????????????????????????????????????????????????????????????????????????? (Fix pytest fail and linting issues.)

ArchooD2 closed this Jun 22, 2025

Conversation

ArchooD2 commented Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

ArchooD2 commented Jun 17, 2025

Uh oh!

ArchooD2 commented Jun 17, 2025

Uh oh!

MuddyHope commented Jun 17, 2025 via email

Uh oh!

ArchooD2 commented Jun 17, 2025

Uh oh!

ArchooD2 commented Jun 17, 2025

Uh oh!

ArchooD2 commented Jun 17, 2025

Uh oh!

MuddyHope commented Jun 17, 2025 via email

Uh oh!

ArchooD2 commented Jun 17, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ArchooD2 commented Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MuddyHope commented Jun 17, 2025

Uh oh!

ArchooD2 commented Jun 17, 2025

Uh oh!

ArchooD2 commented Jun 17, 2025

Uh oh!

MuddyHope commented Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MuddyHope left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MuddyHope commented Jun 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ArchooD2 commented Jun 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ArchooD2 commented Jun 18, 2025

Uh oh!

MuddyHope commented Jun 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ArchooD2 commented Jun 18, 2025

Uh oh!

ArchooD2 commented Jun 19, 2025

Uh oh!

MuddyHope commented Jun 19, 2025

Uh oh!

ArchooD2 commented Jun 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ArchooD2 commented Jun 17, 2025 •

edited

Loading

ArchooD2 commented Jun 17, 2025 •

edited

Loading

MuddyHope commented Jun 17, 2025 •

edited

Loading

MuddyHope commented Jun 18, 2025 •

edited

Loading

ArchooD2 commented Jun 18, 2025 •

edited

Loading

MuddyHope commented Jun 18, 2025 •

edited

Loading

ArchooD2 commented Jun 19, 2025 •

edited

Loading