🔎 BigO(Bench) Sample Explorer 🔎

BigO(Bench) assesses the capacity of Large Language Models (LLMs) to comprehend time-space computational complexity of input or generated code.

Select a task, a problem and a class of complexity for the solution, and compare the outputs of the LLMs!

Homepage Paper Code HF Dataset Leaderboard

Prediction
Generation
Ranking
Time
Space


					

					

						
						
						
					

					

						1 / 5
					

				

				

				

				
				

					

						

					

					

				

				

			

		

		


		

			
			


				
				

					

						

					

					

						
						
						
					

					

						1 / 2
					

				

				

				

				
				

				

					

						

					

					

				

				

			

		

		


		

			
			


				
				

					
					

						

					

					
					
					
					
				

				

				


				
				

					

					

						

					

					

				

				


				
				

					

						

					

					

					

					

					

				

				


				

					

						

					

					

					

					

					

				

				

			

			

			


			
			
		

	


	

		

			

				
📝 Notes

				

					
    
						
  1. 
							Complexity is defined in the following way in model prompts, so to remove some ambiguity
							(here is an example for time, in the case of space, 'time' and 'space' are just replaced by
							one another): 
     Provide the time complexity for the following competitive programming
							question and corresponding solution. When analyzing the complexity of an algorithm, consider
							the worst-case scenario where all possible input combinations are tried, given the following
							conditions: 1. the inputs must adhere to the specified data types of the problem; 2. the
							inputs should not cause the code to crash or exit on an exception; 3. the inputs do not
							necessarily need to satisfy additional constraints that are potentially mentioned in the
							problem statement; 4. calling input() does not consume runtime nor memory, but of course any
							operations on top of it or afterwards will be counted towards runtime and memory footprint;
							5. Anything printed gets added to the memory. You can take advantage of Python-specific
							optimizations provided by the underlying CPython interpreter or compiler to achieve the
							desired complexity, and you must account for them when analyzing the complexity. Please
							ignore any constraints on the input sizes that may have been previously mentioned in the
							problem description. Compute the big-O complexity as if inputs can be as large as possible.
							Output the big-O time complexity only, no explanation needed, no other words needed.
						
    2.