If you’re just beginning to plan your Office 365 rollout (or are in the process of rolling out additional Office 365 applications), it’s important to consider the benefits of running a formal pilot program. A pilot enables a subset of users to access Office 365 prior to its rollout to the entire organization. There are many benefits to running a pilot, including:
- Road-testing your communications and training. I host weekly Q&A calls for my pilot testers. The calls enable pilot participants to provide feedback on their experience, highlight favorite features, identify gaps in our training materials, recommend new methods for engaging business teams, etc.. This feedback is key because it comes from the business. Whenever possible, I like to test out the pilot users’ training ideas during the pilot itself. If they request a What is Office 365 training session, I’ll assemble a training curriculum and offer the new class to the pilot group. Taking this extra step during the pilot enables me to gather more feedback and puts me in the best possible position for my org-wide rollout.
- Building a knowledgeable set of pilot users that can support your rollout and (hopefully) recommend Office 365 to their colleagues. Giving users early access to Office 365 and offering them the opportunity to impact your rollout builds rapport. This enhanced sense of community engagement will help you build momentum for your rollout, enhancing user adoption.
- Testing your license enablement and support processes. As part of your pilot, you’ll need to enable Office 365 applications for your pilot testers. You’ll also need to provide support as they begin using the products. This is an excellent opportunity to test (and improve) your licensing and support processes. After all, pilot testers are more likely to forgive enablement issues and support delays. And learning from your mistakes during a pilot will ensure the same mistakes don’t occur during your org-wide rollout!
- Identifying use cases and success stories. Ideally, your pilot testers will be leveraging the new capabilities they’ve received. Consider setting up time for them to show off their results. You may find unexpected use cases for Office 365 and success stories that highlight key value-adds. Leverage these use cases and success stories to tell the story of Office 365’s business value.
- Validating your governance and compliance policies. A pilot program enables you to see how Office 365 performs in your environment. Your networking team can validate network traffic is flowing smoothly. Your information security team can validate all appropriate risk requirements have been met. If you have compliance or legal record hold requirements, those teams can validate to ensure data is being scanned, stored, and supervised appropriately. If there are issues, you have time to correct before Office 365 is rolled out to your entire organization.
Pilot Planning
But how big should your pilot be? And how long should it last? The scale of your pilot should reflect the size of your organization and the level of governance and compliance controls you need to implement.
As a general rule, I recommend building a pilot program that is roughly 3% of your overall user base. If you’ll be rolling out Office 365 to 10,000 users, that would mean having a pilot group of 300. If you have a user base of 50,000, you’d want to build a pilot group of 1,500. While this 3% target may sound large, it affords the best opportunity for user feedback. Keep in mind that you cannot expect strong participation from all pilot users. No matter how well-intentioned, there is always a percentage of pilot testers that contribute minimally or not at all. Targeting 3% of your user base ensures you will have a viable set of pilot testers to try out your products.
Your pilot program will also enable you to test key governance and compliance controls (e.g. archival and supervision of Outlook email, record retention, SharePoint hub site governance, etc.). In order to run a valid test, you will need a sufficient data sample size. If 3% of your user base will not give you enough data to work with, increase the size of your pilot group accordingly. It is always best to adjust any necessary governance policies or security and archival controls before your org-wide implementation.
Now that you have a rough idea of your pilot size, it’s time to start planning who should be part of the pilot. Ideally, you should target:
- Highly engaged users from across multiple business lines (not just IT).
- Green dot and yellow dot users that are quick to adapt to change.
- Volunteers. Always engage those who want to be part of your pilot. These users are more likely to engage, will dedicate more time, and are much more likely to provide feedback. While it may be more work to amass a set of volunteer resources, they will be more engaged and provide better feedback than voluntold users.
- Strong communicators. Feedback is an essential part of the pilot process. You want to engage those that are willing to provide written or verbal feedback.
- A variety of personality types. Ideally, you want to include technology optimists and pessimists in your pilot group. Technology optimists have a good impression of IT and are generally enthusiastic about trying out new technologies. Technology pessimists have a stronger “what’s in it for me” mentality and need to see or hear something compelling before they decide to jump on board. Incorporating both technology optimists and pessimists in your pilot will give you the best opportunity to validate your Office 365 messaging and training.
How long should your pilot last? While many companies have intense pressure to roll Office 365 out quickly, I’m a firm believer in meaningful pilots. The more time spent piloting and refining your approach to governance, training, and license enablement, the more successful your rollout will be. A pilot period of 3 weeks is incredibly tight, but can generate value. A pilot period of 1-2 months will generate more user data and enable you to refine your training offerings.
There is one exception to this “lengthier is better” rule for pilots. If you work for a company with a strong technology innovation culture and most of your users are very comfortable with technology change, a longer pilot may not be necessary. If your organization is facing a great deal of technology debt in the workforce productivity space, a longer pilot will better enable you to build momentum for the upcoming change.
Pilot execution
Now that you’ve completed your pilot planning, it’s time to execute your vision. Plan to launch your pilot with a formal series of kickoff meetings, brown-bag lunches, and/or targeted pilot communications. The goal is to celebrate this important stage of your Office 365 rollout efforts. And the more positive noise you can generate, the more pilot user engagement you’ll see.
You’ll also want to make it easy for pilot users to learn about Office 365 and provide feedback on their experience. A few ideas I’ve seen work well:
- Share “getting started” scenarios. Many users may feel intimidated when they first open a new Office 365 app. Providing quick, easy-to-follow “getting started” scenarios for each app gives your pilot users a running start. If you’re going to pilot OneDrive for Business, for example, you could create getting-started scenarios that explain how to:
- Create files in OneDrive
- Save Microsoft Office files to OneDrive
- Share a OneDrive file (or folder) with someone
- See OneDrive files that have been shared with you
- Open files using the OneDrive sync client
- Edit files in the OneDrive mobile app
- Host weekly Q&A sessions. As I mentioned previously, I host weekly Q&A calls for my pilot testers. The calls enable pilot participants to provide feedback on their experiences and ask product-related questions.
- Schedule “Show & Tell” events where pilot users can share Office 365 tips and demo solutions they’ve built. Provide an opportunity for your pilot users to shine. Schedule a recurring meeting where the pilot users come together to share cool new tricks they’ve learned and demo solutions they’ve built in Office 365. For best results, keep this a peer-to-peer sharing meeting. Having a pilot user demonstrate a new Microsoft Flow they’ve built is powerful stuff. Their excitement and confidence in using Office 365 will motivate other pilot users to follow suit.
- Build a private Yammer group for your pilot users. Yammer is a great “thinking out loud” app that supports discussion and sharing of ideas. I recommend creating a private Yammer group where pilot users can share insights, ask questions, etc. Add all pilot testers to the group before your pilot begins and send them the group URL for easy reference. As the pilot coordinator, it’s important you actively participate in the Yammer group. Share Office 365 tips, provide links to appropriate training resources, and answer pilot user questions.
- Track your pilot user Yammer sentiment. If you’re providing your pilot users with a Yammer group for sharing ideas, use Microsoft Flow and Azure Cognitive Services to perform sentiment analysis on the pilot Yammer posts. The sentiment data gathered may provide insights into your pilot group’s overall satisfaction with Office 365 and help you identify solution use cases.
- Stay in touch. I always want my pilot testers to feel like they are part of a valued community. Connect with pilot users that are in your geographic area by hosting meet-ups or coffee chats. Communicate with geographically-distributed pilot users via Yammer or Teams. And share an “Office 365 tip of the week” for all pilot users.
Want to learn more?
Asif Rehmani published a great article on the key reasons to include an early adopter program in your Office 365 rollout. Check it out–it’s well worth a read!
The guide also shares the inspirational champion story of the Best Buy SharePoint Ninja program. I’m thrilled to see the program still being upheld as a success story. Launching the SharePoint Ninja program at Best Buy was one of the highlights of my career, and effectively-built champion programs are a vital part of the Microsoft adoption story. For more information on the Best Buy SharePoint Ninja program, check out the session 
Sarah Haase 