Looking for good resources on starting a beta team and gaining in quality assurance

#1

Heya guys,

I’ve been developing WordPress products for 5 years now. For those of you who’ve taken a shot at product development (alone or with help) you know how multifaceted the attention requirements are.

One person has to be a marketer, a customer service rep, a graphic designer, a product developer, a coder, a quality control developer, and more all at once. If you have a partner you can divide up tasks and responsibilities, but even then it’s difficult for two to get the job done the way a larger company would.

It’s hard. It’s humbling. And success isn’t guaranteed.

For the past three years I’ve been building the components that power Inbound Now, which is a SAAS in development that attempts to bring a comprehensive Inbound Marketing Suite similar to HubSpot directly into WordPress. I’ve made incredible strides. I’ve good review and poor reviews. I have a partner. He’s great. He provides development assistance and hunts out new technologies, goes to conferences, manages the front end sales site, controls company direction with me, etc. Without him I would be years behind and a poorer developer.

In my opinion all is moving well, but I have pressing needs and worries. I want to provide my customers with a top quality product and I am noticing that I am failing a handful of them with every new release.

We clean up the mess quickly, but ideally the mess should never make it to our users instances.

My ideal response would be to have a new partner on my roster that is 100% focused on quality control and what their primary responsibilities would be:

1. Created unit tests and front end macro tests for every customer support issue that ever comes up

I’m talking about a person who knows use macro programs like ubot or other front end automated testing suites AS WELL as Travis CI type unit testing which I’ve never found time or understanding to write myself. Then every time a new support issue arises then the QA person would solve it and analyze it to see if a test can be created to assure the solution always stays in place.

Then every time new updates arrive through git the unit tests would run AND the front end macros would run.

That would take finding and hiring the talent, or having another partner join in exchange for equity and cover this responsibility.

The alternative and more immediate solution is to develop a Beta team. Which I’ve never done. And it seems inefficient because I can’t imagine users that would want to be early adopters to a release that might have undiscovered bugs in it. But apparently there are people out there like that.

Which brings me to a question:

What are good resources for learning how to develop a beta team? What are the insights on developing beta teams?

So in recap this topic states:

  1. Inbound Now’s deployment systems needs work on the quality control front.
  2. I imagine what a quality control partner would be like and dream on his/her arrival of talent.
  3. I wonder on educational resources about developing a beta team, and wonder if the community has any valuable advice/information/education on starting up and maintaining a beta team.

Thank you wp developers for listening. I’m sorry if I hit the TLDR mark!

#2

Hi @metacaptin I’ve just started learning unit testing, so I’m not your ideal person right now but if you cannot find anyone, in the next future I may be able to help you. Just write down my name :smile:

1 Like
#3

It’s not exactly the same thing, but I always think of what Pippin says about building a team around Easy Digital Downloads:

“Ask for help”

The first step is to ask your customers whether they’re interested and maybe what they would want to get out of it. If your particular product doesn’t offer a lot of incentives for testing a beta service, then think about ways to incentivize it: special discounts, unique reports, community credits, a greater say in future developments, etc.

Slide 20 and onward in this talk from Pippin includes lots of basic tips. He’s talking about code contributions, but I think a broad takeaway that would apply in your situation is about giving people buy-in and a rewarding sense of participation.

And a more technical recommendation from user experience with other services: make it possible to quickly jump from stable mode to beta mode. That kind of “no risk” exposure in services always tempts me to take a look at upcoming changes.

2 Likes
#4

While I cant advise on finding a new partner, I’ve some advice for changes you could make with minimal effort.

Do you currently have unit tests for any areas of your code? If not prioritize. areas you’ve found to be causing the most issues are a good start, your goal here is to improve your “teat coverage” if you have 50 functions. Aim to create tests for maybe the main 5 problem areas.
If you’re adding a new feature, create tests during development.

As for creating a beta team, I’d refer to this as a QA (Quality Assurance) team. Some members may be users of the plugin who are not developers but have enough skill to install and use your plugin.

Is your free version on GitHub? I’d recommend adding it if not. And advertise its precence on your plugin. If technically minded people find issues and their fixes they’ll usually be more than happy to share, especially if you’ve made it easy to do so!! These people form part of your QA team, through the power of open source! This may highlight a user whom you think fits the bill as a partner and get talks about such partnerships going.

I hope this point help. Apologies for any spelling, I’m currently on a bumpy bus!

1 Like
#5

Still reading replies,

I like the idea of being able to quickly switch from beta mode to current release. It would be incredible for there to be a toggle to select/load the version tool you would want to run.

As developers we tag our versions in the WordPress SVN so there is always a historic copy of the last version available.

A plugin could be developed that allowed users to quickly load version tags from WordPress’s SVN; for all plugins not just developer specific plugins.

Then we could create documentation resources for our users on how to roll back software if they are experiencing bugs and the major problem of momentary suffering for our early adopters would be solved.

On the other hand we serve development files from GitHub so, whatever plugin was developed to let users roll back or roll forward plugin versions should be extendable to add in Github connections for certain plugins. This would allow us to let out users switch to ‘developer’ mode very quickly which would be also great for customer service solution testing.

#6

@darren We do have some unit tests, very basic ones that help us make sure our plugins do not fatal error on activation.

We do have a github and have our call to actions in place to tell other developers about them. We’ve even had a couple of merge requests and user issues raised. What’s cool is that usually we see our customers raise issues with us on our support forum, and we ask them to make a merge request, and then they send their techies to do the work and make the merge request. Noone’s wrote a unit test for us though yet. TBH my code is not verry pretty for our Landing Pages plugin and that’s our most used one. Only in later products (Calls to Action and our unreleased Email Component & Marketing Automation component) did I start writing beautiful, well documented OOP code… which two of those three components have yet to be released.

A lot of the successful phenomenon surrounding Pippin’s code base over at EDD, I believe, is a result of his high quality practices from the beginning and him being well networked with the WordPress.org development crowd which are well trained folk. And his inclination to creating educational pieces for guys like me.

Comparatively I started very poor in understanding of coding standards and have improved dramatically over a short period of time creating an portfolio array of mixed standards that can be unappetizing when looked at as a whole (and not much can be done about besides delegating time to refactoring as time becomes available). Pippin has actually been a huge educational leader for me.

This is a reason my project also needs to win another Lead Developer (separate from the QA developer), so we can get our entire portfolio up the OOP standards, which will make it easier to write unit tests for and easier for our developer users to contribute to our codebase.

#7

The problem with this is that many plugins perform data migrations during updates. It’s not always as easy as just rolling back a set of files. The principle idea is really great though, and if someone’s site has broken after an update, installing a previous version is probably the first thing you would try.

#8

You are right. I had not thought of that. Even we do data migrations with our tools. If we were to implement this feature, rollbacks would not work when data migrations existed between version releases.

#9

I’ve still started a foundation for version switching ( WordPress SVN only for now). It will work for all wp plugins that tag their releases:

Next step is to create a mod that hooks into a GitHub branch and allows toggling between a ‘development’ or ‘beta’ branch and the latest stable SVN tag release for a specific plugin. What do you think? And do you think, from a UI perspective, I should make the switch available in the plugins area or should I nestle it into the plugin’s settings page?