I am already doing something on these lines. The big disadvantage of, at least playwright, is that it can take a lot of time for testing, making your loop longer. I guess you also notice this issue. A great solution I found for this is, making a cli first architecture, and then it is simpler to test. The UI only is a thin layer of visualization. If you can move 100% of the features into the cli, your web tests will be minimal, because the features will be fully end-2-end testable via the cli.