AI Challenge Q&A

Is there a preference for less complex projects that would deliver an end product in a shorter timeframe and/or consider the competition award as ‘seed funding’ for a longer term for a longer-term project – less likely to be a showcase project in the short run?

No, there is no preference for less complex projects, we are happy to receive and consider applications with ideas/use cases that are complex, or conversely those that may appear less complex. Each application will be considered on its merits in relation to the judging criteria including elements such as innovation, feasibility or scalability. We may, however, for those ideas shortlisted to the proof-of-concept stage, focus on a specific aspect of the idea/use case in the more complex use cases, to create something demonstrable to bring the idea to life.

How focused should the application be? We have 2-3 ideas/functions that could be built into one product or separately.

Ideally the application and idea should be focused on a single use case, as that will likely make it more straightforward to assess and to progress the proof-of-concept stage, should the application be short-listed. However, scalability and wider potential of the idea/use case for broader application and adoption, both within your organisation and potentially across the wider public sector, are elements of the judging criteria. So, we welcome ideas/use cases that could be scaled into larger applications (or products) and adopted wider. Please articulate that in your application.

Should the proposal be applicable across all public sector organisations, or can it focus on challenges from organisations focused in a subject-specific field?

We anticipate that most applications will be from single organisations focused on a subject specific use area and use case. However, we are also keen to receive applications with ideas/use cases that have wider potential, both within and possibly across the wider public sector, and have the potential to scale. This is an element of the judging criteria. We are also happy to receive and consider joint applications, involving one or more public sector organisations, who wish to collaborate in addressing a problem or opportunity through the use of AI.

Should the proposal be specifically around the application of AI, or is there opportunity to explore more creative applications, i.e. genAI?

We are happy to receive applications to explore any AI use case including one that would include generative AI.

Does Storm ID have a 2-minute "elevator pitch" summary of a successful AI implementation they've recently implemented with a local authority?

Storm ID have been developing AI solutions for over five years across government and health sectors.

We have developed AI solutions for Local Government organisations including a chat solution aimed at citizens, for managing bin schedules.

Further to this, we have implemented a range of AI solutions with a number of public sector organisations. Some examples below:

UK government department where we used generative AI to extract analytical insights from large sets of structured data and unstructured data (or semi structured documents) data related to programme performance. This involved a high degree of prompt engineering to enable the extraction of very useful insight.

For another public sector organisation, who manage historic sites across the UK we used AI to analyse images of sites and locations for classification purposes and to enrich their dataset with additional metadata for images. We also analysed unstructured data about sites and images, creating text-based summaries for presentation to users via their website.

Is the aim of the proof of concept to design and build a working prototype (e.g. something that can be run, tested, and interacted with)?

Yes, the aim is to create a demonstrable prototype through a combination of workshop and hackathon development activities that will bring the idea to life. The expectation is that the prototype will be able to be run, tested, and interacted with.

Can you please clarify this part of the application form - “any assumptions being made about the application of AI technology. For example, would the ide involve pre-trained machine learning models, Generative AI and Large Language Models (LLMs), or Custom AI Models.

We’d like to be clear that details of specific AI models are not required to be included in your application. But if you do have any assumptions or expectations of specific models that you think are relevant to your idea/use case, then you should feel free to suggest within your applications.

This will be more relevant in the proof-of-concept stage when, for the short-listed ideas, a model assessment will be carried out to compare and select one or potentially more than one model, which Storm ID will coordinate as part of the process collaboratively with you. We will be considering a range of code hosted and open-source models and selection will be made against a range of criteria including functionality, performance, robustness, and ethical considerations.

Does Storm principally use resources from one of the big 3 (AWS. Microsoft/Open AI, Google) or do you take a more hybrid approach?

We have the capability to use both AWS and Microsoft Azure, including OpenAI via Azure OpenAI. So there is flexibility to select the right platform and tools depending on the model operation and data processing requirements, between the AWS and Microsoft options.

We would probably not consider Google Cloud unless there is a very strong case for it.

Depending on the problem in focus, and hence the type of algorithm or pre-trained model required, applicants might need access to graphics processing unit enabled machines. Are you supporting access to machines?

Yes, we would provide access to those machines and infrastructure, which would be GPU enabled or virtual machines or other related infrastructure as part of the PoC.

One key challenge within the Scottish Government is the lack of common deployment infrastructure. What are the plans to ensure that after the successful development of a deployable solution, it is deployed - especially if it is an item that must be deployed in-house due to sensitivity/confidentiality reasons?

We understand the challenge. For the proof of concept process with the shortlisted applications, we will provide the environment for the development and demonstration of the prototype. So, this question is probably more for the winning organisation and the potential future development and deployment of a production solution, which we would be happy to provide guidance on at the appropriate point, considering aspects such as containerisation.

Do you aid the development of the drift monitoring and continuous/regular model re-training aspects?

Yes, that could be considered if it is relevant for the use case. There are different approaches, one of which is the auditability of ongoing model performance perhaps using a systematic evaluation within the use case, or a more user-centred approach to performance evaluation where users are given the opportunity to “score” in a seamless way the results being provided which are then incorporated in a feedback loop. We would definitely have a human-in-the-loop to review and contribute corrections where that fits in with the use case.

We would certainly see a need for some form of continuous/regular model re-training as we’re aware there’s potential for performance to drift or decrease over time, to we would look to put several mechanisms in place to mitigate that happening.

Given the challenge is targeted at the public sector, have the Public Procurement Regulations been considered?

Yes, the public procurement regulations were considered as part of due diligence in the planning stage of the challenge. We also consulted with a range of public sector organisations and depts to canvass views and input on the challenge and how we planned to offer and run it, including Scottish Government Procurement Directorate’s Procurement of Innovation team. Based on the due diligence, we concluded there were no barriers to such a challenge, which encourages innovation and does not require any financial commitment to participate, and there is ample precedent of similar challenges that involve public-private sector collaboration across UK government.

The AI Challenge is therefore a pre procurement activity. For further information on the Scottish Government procurement of innovation journey please see Innovation Research, Pilot, and Innovation Register

We would also encourage you to consult your internal procurement or legal teams for any further guidance on procurement considerations, intellectual property, or data protection, especially should you be successful in being shortlisted or ultimately winning the challenge.

Who owns the intellectual property of the end product?

Throughout the process we will ensure that participating organisations intellectual property is protected in terms of confidentiality of your idea and the information you share with us. For those organisations participating in the proof of concept stage and beyond, we will put in place a formal statement of work with sufficient terms and conditions that protect your foreground intellectual property.

Who are the judging panel members?

We are in the process of finalising the judging panel, and we have a number of prominent domain experts across a number of relevant areas, who have an interest in the adoption of AI in the public sector who have already agree to participate. We expect there to be a maximum of 5 judges and we will announce who they are shortly.

Is the challenge prize of £20,000, an in-kind contribution of technical advice? Or is it a grant which the winner can use to develop the solution?

Yes, it is an in-kind prize, not a financial grant that the winner can use at their discretion. Specifically, it is £20,000 of consultancy or technical services from Storm ID that can be used flexibly to further develop your idea or proof of concept towards fruition. We will discuss and agree with the winning organisation how best this could be utilised.

If that £20,000 prize in consultancy services was broken down - at ball park how many hrs would that equate to?

The £20,000 prize in consultancy or technical services would equate to c. 30 days or 220 hours of services, depending on the breakdown of service types required to complete the agreed scope of work.