Erros with --xml-report behavior

[sean-petal]

Describe the Bug

The --xml-report option fails when the output report includes folders. It reports
ERROR: --xml-report must be a filename, not a path

for:
pytest --generate-xml --xml-report report_output/final_report.xml

While
pytest --generate-xml --xml-report final_report.xml

works fine.

Note: the documentation page is also incorrect -- it passes True to --generate-xml argument (which it can't handle) and then includes an example with a folder which fails. (link: https://github.com/reporterplus/pytest-html-plus/blob/main/docs/cli/generate-xml.rst)

To Reproduce

See description -- just run with a path in the filename for --xml-report

Command used:
See above

What actually happened:
ERROR: --xml-report must be a filename, not a path

Expected Behavior

It would allow me to specify an xml output inside a subfolder for better organization.

Environment

pytest-html-plus version	1.1.0
pytest version	9.0.3
Python version	3.12
OS	Windows 11 or Linux (CI/CD)

[reach2jeyan]

@sean-petal
Thanks for reporting this. The current behavior is intentional: --xml-report accepts only a filename and places the XML file inside the directory specified by --html-output, keeping all report artifacts together. We previously moved away from supporting independent paths to avoid output files being scattered across multiple locations. However, the documentation examples are incorrect and will be fixed.

If you'd like the XML to be written to a different folder, you can specify that folder via --html-output, for example:

pytest \
    --html-output myownFolder \
    --generate-xml \
    --xml-report myownxml.xml

This produces:

myownFolder/
├── report.html
├── myownxml.xml
├── screenshots/
└── ...

Could you share your use case for having the XML file in a different location from the HTML report? Understanding the workflow will help determine whether support for custom XML paths would be useful as an enhancement.

[sean-petal]

High level, I'm trying to add to my bitbucket pipeline to automatically rerun failures (pytest-rerunfailures) but bitbucket only sees it as a success if it reruns, so I hoped that pytest-html-plus would include the details of the reruns and let bitbucket track the flaky tests. I think I was naive to think bitbucket could handle that :)

I original found this issue because I have a method where I capture the logs to pass them to another step for reporting to bitbucket. So it tripped me up because I couldn't specify where the files went. Once I realized exactly what you said above (it forces output to a specific folder) I just added a script step to move the files. I could have use the --html-output instead.

The main thing is most tools like this (particularly pytest) let you specify where a file goes if you tell it a filename. I understand what you did what you did, and it would be ambiguous if no path dropped it in the standard folder instead of in the current directory.

If you update your docs then I think all is good!

[reach2jeyan]

Thanks for the additional context and for explaining the Bitbucket workflow.

Your expectation makes sense, as many pytest plugins allow paths to be specified directly. Since the recommended approach of using --html-output solves the problem for you, I'll treat this primarily as a documentation issue for now.

I'll keep the idea of supporting custom XML paths open and explore how it could be implemented in a way that remains intuitive and avoids confusion around report artifact locations. If we decide to pursue it, I'll tag you on the enhancement ticket.

Thanks again for the feedback, for sharing the use case, and for using the plugin!

[reach2jeyan]

High level, I'm trying to add to my bitbucket pipeline to automatically rerun failures (pytest-rerunfailures) but bitbucket only sees it as a success if it reruns, so I hoped that pytest-html-plus would include the details of the reruns and let bitbucket track the flaky tests. I think I was naive to think bitbucket could handle that :)

Also, since you mentioned reruns and flaky tests, pytest-html-plus already exposes rerun information in the JSON report. The summary section includes a flaky count, individual tests identified as flaky are marked with "flaky": true, and each test entry includes an attempt_count field. An attempt_count greater than 1 indicates that the test was rerun, which may be useful if you're consuming the report programmatically or integrating with a custom Bitbucket reporting step.