AB Testing

The Test

During the A/B testing phase of AI-powered travel itinerary application, I gathered detailed qualitative feedback from  participants (P1-P5) who tested 2 distinct versions of the application.

Th invaluable qualitative data was complemented by quantitative insights. Specifically, 4 desktop participants engaged with Version 1 and Version 2. For mobile testing, 2 participants interacted with Version 1, and Version 2. This comprehensive approach allowed m to gain a holistic understanding of user experiences and preferences across different platforms and versions.

The Insights

The testing revealed several key insights. Participants generally had positive initial reactions to the application, finding it helpful, useful, and easy to use. The visualisation of travel plans notably created a sense of comfort and eased travel anxiety.

Regarding version preferences, more participants expressed a preference for Version 2, primarily due to its better information and transparency, which in turn enhanced their trust and confidence.

Furthermore, specific Explainable AI (XAI) strategies such as transparency, step-by-step reasoning, confidence ratings, and semantic labels for activities were highly appreciated, helping users understand and trust the AI's recommendations.