# arcagi
A Rust attempt at the ARC-AGI prize 2024
WARNING:
-----
This code is messy and imcomplete and is likely to change rapidly. At the
moment it is a set of experiments to guage what works well and what does not.
It is perhaps 20% complete and will answer about 20% of the examples given.
If you use this code as part of a Kaggle submission, or in any other way, then please give credit where credit is due.
Background
----------
The Kaggle ARC-AGI challenge 2024 is an attempt to get computers to solve
visual 2D puzzles, similar to those in many IQ tests. To solve this challenge
in a principled way requires perception of stucture and reasoning guided by
constraints. It currently cannot be solved by LLM/GenAI based methods though
there may be a small place for them (or possibly just simple Neural Networks
for specific recognition tasks).
This code is a first submission for the prize, it could conceivably solve all
the puzzles in the challenge, but at the moment solves just over 10% in the
public evaluation set. Much more work is required to reach the 85% or better
level.
This prize in important because it will demonstrate reasoning abilities and
not just the memoristion with some 'light' generalisation that is currently
demonstrated by all Gen-AI models (in fact all Machine Learning models).
Abstraction, Generalisation and Functional composition are key concepts.
These work in tandem with constraint recognition and discovery to tame the
size of the search space.
This is a pure Symbolic AI approach and hence is deterministic. There may be a
place for an LLM/Gen-AI in object recognition and/or generating candidate
solutions.
It was hoped that the author's previous experience of LLMs (see LLMClient),
would help (for code generation). It might eventually, for now there is
plenty to explore with the current Symbolic approach.
Code generation was experimented with, given some data
structures and a textual description of the problem, but the code
generated was poor quality and time consuming to debug (Rust is probably not
the best target for code generation given it's memory management as simple
next token prediction does not understand that).
See: https://www.kaggle.com/competitions/arc-prize-2024
See inet submission.
To run this the first parameter is one of 'training', 'evaluation' or 'test'. There is no solutions file for the test run, this is expected so ignore the error message. A second parameter can be supplied which is the test to run, useful
for testing individual cases. A typical call to run the training set would be
'cargo run --release training'.
Kaggle notebook
---------------
```
from cffi import FFI
ffi = FFI()
ffi.cdef("int test();")
rust = ffi.dlopen("/kaggle/input/release/libarc_agi.so")
a = rust.test();
print(a)
```
If you wish to use this for a Kaggle submission, then please ask the author
who will be happy to share a Kaggle shared object. Credit is expected of course..
NOTE
----
To compile the above for kaggle a docker image needs to be created and the .so
file uploaded and referenced, somewhat tedious. It will run locally for public
data test with cargo as normal. A release version should run in less that a
second on most machines, this will change as more cases are completed.
22 December 2024 - After o3 'Success'
-------------------------------------
The three o3 unsolved puzzles are solved by this implementation. c6e1b8da, 0d87d2a6 and b457fec5.
I'm not happy with 0d87d2a6 as the extending arm should only change other shapes it crosses, if and only if it goes through them, not a near miss. That's the 'best' human interpretation! I appreciate 2 attempts are allowed, though I don't think that should be necessary, the interpretation should be deterministic given the experimental examples!!! Humans solve IQ tests analytically, not by thinking, 'Ah, this is ambiguous, these are the choices'. Or guessing, though as a last resort we have all done that, but it's not repeatable and not appropriate here! Can we remove this 2 guess approach please for version 2.