Discussion about this post

User's avatar
Eric F's avatar

Great write-up! I have found similar gains myself, and have also faced some of the shortcomings.

Having the LLM write tests can be a time saver, but I sometimes find that while it writes descriptions for what would be impactful tests (it(‘does thing’, …)), sometimes the test steps themselves don’t actually test the thing being described, so I’ve started adding the instruction to have the agent make sure the steps within the test case actually test what the description statement claims to.

Expand full comment
1 more comment...

No posts