Overview of changes
Here is a list of changes made since the IOI 2007 competition rules were first published on June 18:
- When detailed feedback is enabled for a task, you may choose which 10 submissions will get detailed feedback. Previously it was always your first 10 submissions.
- The exact amount and type of whitespace you use to separate the elements in the output file will not matter in most cases. However, it is recommended to follow the output specification as precisely as possible.
Here is an overview of changes in the competition rules between IOI 2006 and IOI 2007:
- Contestants will not use comment tags in their source to identify the task. They will select the task while submitting the solution.
- Batch tasks will not read from or write to any files. Standard input and output will be used instead.
- The submission will be accepted for grading if the compilation was successful. The submission will be accepted even if the program does not solve the sample tests correctly.
- WindowMaker will not be available on competition computers. XEmacs, Xwpe and Lazarus will be available.
- Different compiler options.
- Contestants may not bring their own CD-s for the practice round. They may only bring printed solution on paper.
- Contestants may submit clarification requests during the first TWO hours instead of one hour.
- A clarification request will be answered with one of the following: "YES", "NO", "ANSWERED IN TASK DESCRIPTION (EXPLICITLY OR IMPLICITLY)", "INVALID QUESTION", "NO COMMENT".
- Printing limits are defined in a different way.
- When using the TEST interface, the submitted input files must not exceed 2MB. Contestants may, however, compress input files using bzip2.
- If a contestant submits more than one solution for the same task, he or she may select one of them as the active submission, which will be graded.
- A few smaller typing errors are corrected; some of the text is clarified or simplified.
IOI 2007 Competition Rules
These Competition Rules cover Competition Procedures and Judging
Procedures. These Rules are not final, changes may be made, and the
final version will be distributed at the first GA meeting of IOI
Delegation Leaders have the responsibility of ensuring
that all members of their delegation fully understand these rules
and abide by them.
All of the tasks in IOI 2007 are designed to be algorithmic in nature. Efficiency plays an important role in some tasks. Whenever efficiency of an algorithm is important, test cases will be designed to differentiate between solutions with different levels of efficiency; the smallest test cases will be designed so that any reasonable correct solution for the tasks can solve them within the task constraints. Therefore, it is worthwhile to attempt a task even if the contestant does not know how to solve large test cases within the constraints.
Tasks may be of the following types:
Solutions comprise a single source file of a computer program which reads data from the standard input (stdin) and writes its answer to the standard output (stdout).
Solutions comprise a single source file of a computer program which interacts with an "opponent" program provided by the organizers using the standard input and the standard output for reading data and writing results.
Solutions comprise a set of "output" data files.
For every task a memory limit will be specified. This limit is on the overall memory usage including executable code size, a stack, a heap, etc.
The source program provided by the contestant must be contained in a single source file. The task statement will define:
- the task to be solved,
- the input and output data formats,
- the input value ranges (when applicable),
- the resource limitations for the computations (e.g. CPU time, memory limitations),
- elements of the grading criteria (e.g. "40% of the official test data will have N up to 100"),
- any other constraints on the program.
The source program provided by the contestant must be contained in a single source file. The task statement will specify:
- the task to be solved,
- the input and output data formats,
- explanation of how to interact with the "opponent" program,
- the resource limitations for the computations (e.g., CPU time, memory limitations),
- elements of the grading criteria (e.g. "40% of the official test data will have N up to 100"),
- any other constraints on the program.
The competition might include tasks for which input data is given to the contestant and the contestant is required to produce only the output data as a solution. If the contestant writes programs to help determine the output data, the programs must not be submitted with the solution. The input data will be provided in ASCII text files. For these tasks, the task statement will specify:
- the task to be solved,
- the structure of the input and output files,
- information on obtaining the full set of official input files,
- grading criteria.
Input and output data
In all tasks, input and output data consist of one or more lines,
each ending with an end-of-line character, including the last
line. The end-of-line character in Linux is represented by the
single character '\n' (ASCII code 10). Each line contains one or
more space-separated items. An item is a string of printable
non-white-space characters (ASCII code from 33 through 126,
inclusive). An item may represent an integer or a general string;
the meaning of each item will be given in the task
The format and limits for input and output data will be specified in the task description.
The output data should be formatted strictly according to the task-specific instructions.
The exact amount and type of whitespace you use to separate the elements in the output
file will not matter in most cases. However, it is recommended to follow the output
specification as precisely as possible.
There will be two competition days. On each competition day contestants will be given three tasks to complete in the five hours. The schedule will be published on the official competition web site.
There will also be a two hour Practice Competition prior to the first competition day. The purpose of this practice round is to familiarize all participants with the environment and procedures. The practice tasks will be distributed in advance of the IOI. Contestants may bring printed solutions to the practice tasks, on paper only. All contestants are encouraged to take part in the Practice Competition round.
In order to protect the confidentiality of the tasks, all direct and indirect contacts and communication between contestants and delegation leaders are prohibited between the moment where tasks for a competition day are presented to the members of the GA and the end of the five-hour round on the following day. During this period the contestants are not allowed to communicate by any means, direct or indirect, with any member of the GA or anyone who has seen the tasks. The GA members are not allowed to communicate task-related information to anyone who is not a member of the GA. The contestants, the GA members and anyone else who has had access to the tasks must obey any instructions which restrict their access to specific parts of the IOI venue.
If a contestant violates the quarantine, he or she can be subject to disqualification. If some other person associated with a national delegation violates the quarantine, then all contestants of that delegation can be subjects to disqualification.
In order to eliminate any suspicion of cheating, contestants and team leaders should not use mobile phones and should switch off all wireless facilities in all devices during the quarantine.
Competition Equipment and Environment
All contest machines will be installed to boot in Linux. Microsoft Windows will not be available in the contest machines. The following packages will be installed:
- Fedora Core 6; KDE; GNOME;
- web browsers: Firefox, Konqueror;
- editors: mcedit, joe, vim, kate, kwrite, kdevelop, emacs, xemacs, rhide, xwpe, lazarus;
- compilers: gcc >= 4.1, g++ >= 4.1, Free Pascal >= 2.0.4 with documentation;
- debuggers: gdb, ddd;
- STL documentation.
- Chipset: Intel Q965; CPU: Core 2 Duo E4300 (1.8GHz, 2M, 800MHz FSB); RAM: 512 MB DDR2
In order to translate tasks, team leaders will be provided with computers with a version of Microsoft Word with some multi-language support. In addition, Ethernet ports will be available for leaders bringing laptops.
In the competition room, blank paper and writing utensils will be provided. On the competition days, contestants may not bring anything into the competition rooms, but:
- reasonable jewelry,
- writing utensils,
- simple wristwatches,
- keyboards and mice (see below),
- small mascots (see below),
- English dictionaries (see below).
If a contestant wants to bring a keyboard or mouse, small mascots or English dictionaries to the competition, these must be submitted to the technical staff during the practice competition day. Any of these will be checked and, if cleared, will be given to the contestant in the first competition day. After the first competition day, the contestant must leave these items on his or her workstation if he or she wants to use them during the second competition day. Once again these will be checked and, if cleared, will be given to the contestant in the second competition day. After the second competition day the contestant must take any of these items with him or her.
Any attempt to bring any other item into the competition room will be considered cheating. In particular, during competition rounds it is strictly prohibited to bring:
- any computing equipment (e.g., keyboards, mice, calculators, organizers, PDAs, computers, calculator or "data bank" watches),
- any books, manuals, written or printed materials,
- any data storage medium (e.g., diskettes, CD-ROMs, "pen drives", compact flash/smart cards, micro-drives),
- any communication devices (e.g., mobile phones, radios of any sort).
Any electronic or printed materials provided by the organizers during the Competition Round may be used by the contestants (e.g., a Users Guide to the Contest System, or any electronic documentation or reference manuals provided in the installed contest environment or on the provided Contest Server).
Starting the Competition
On the competition days, immediately after breakfast ends, contestants will be admitted to the competition hall. All contestants must wear their ID badges during the competition. Each contestant will have a pre-assigned workstation; this information will be posted in the morning of the competition day. Contestants should be in their seats by at least 5 minutes prior to the start of the competition. On the way to their computers contestants will be checked to verify that they are only bringing allowed items into the competition hall. Small mascots and dictionaries must be submitted to the technical staff during the practice competition day, check Competition supplies section. Contestants must find their assigned computer, sit down, and wait for the competition to begin without touching any:
- other competition materials including tasks.
The beginning of the competition will be marked by an audible start signal, after which the contestant may use any of the above items.
During the first two hours of competition, contestants may submit written questions concerning any ambiguities or items needing clarification in the competition tasks. Questions must be submitted on the provided Clarification Request Forms, expressed either in the contestant's native language or in English. If required, delegation leaders will translate their contestants' questions into English after they are submitted and before they are sent to the Scientific Committee. The Scientific Committee will respond to every question submitted by the contestants. Since this might take some time, contestants should continue working while waiting for the answer to their questions. Contestants should phrase their questions so that a yes/no answer will be meaningful. Questions will be answered with one of the following:
- "ANSWERED IN TASK DESCRIPTION (EXPLICITLY OR IMPLICITLY)" - The task description contains sufficient information. The contestant should read it again carefully.
- "INVALID QUESTION" - The question is most likely not phrased so that a yes/no answer would be meaningful. The contestant is encouraged to rephrase the question.
- "NO COMMENT" - The contestant is asking for information that the Scientific Committee can not give.
Contestants will not be involved in or exposed to discussion regarding their questions. However, in cases where the Scientific Committee feels that the contestant does not understand the task statement, English text which helps clarify the task may be added to the response, and if necessary, translated by the contestant's Delegation Leader into the language in which the question was originally posed.
The IOI competition takes place in a distributed environment in which each competitor has his or her own workstation for development and initial testing. Grading and evaluation take place on the Competition Server, which provides a similar execution environment to that of the contestant workstation.
Each competitor's workstation has only Linux installed. A set of software development tools will be available. These tools are used to develop solutions to the competition tasks. There is no restriction on the number of times a program may be edited, compiled, and run on the workstation.
The workstations will have network access to the Competition Server, to a web server which provides documentation, to relevant task data, and facilities such as printing, test execution, and solution submission.
Facilities provided by the Competition Server are described in a separate document which will accompany these Rules. Both these documents will be available online on the Competition Server.
The workstations will neither have access to each other nor the Internet. Any attempt to access another competitor's computer or the Internet will be considered cheating. Attempts to tamper with the Competition server, such as probing the server with customized URL's, will also be considered cheating.
Contestants are allowed to print only text files during the competition. After a contestant requests that a document be printed, the support staff will deliver the printout to the contestant; contestants should not leave their computer to find printouts. Printouts will be delivered as quickly as possible, but large volumes can produce delays in delivery. The limit on the length of a printed document is 1000 lines. If a document to be printed is longer, only the first 1000 lines will be printed. There is also a hard limit of 50 KB on the size of the document (larger documents will be rejected). On each competition day the contestant can print at most 30 documents.
Contestants submit solutions to the Competition Server via a web browser.
When submitting their solution, contestants have to choose the task and specify the programming language (where applicable). The contest system also provides an option to auto-detect the programming language from the file extension.
Solutions for tasks may be submitted at any time. If a contestant submits more than one solution for the same task, he or she may select one of them as the active submission, which will be graded. By default, when a successful submission is made, it is automatically marked as active.
For tasks which require output files as solutions, contestants must submit each output file separately. The submission facility will validate the format of each output file submitted, accepting only valid output files for grading.
For tasks that require programs as solutions, the submission facility will accept C, C++ or Pascal programs, verify that the program compiles and obeys the stated limits on program source size and compile time. The submission facility will then run the program on the sample test cases given in the task description, enforcing the relevant run-time resource constraints, and report the results to the contestant.
Each submitted source program must be smaller than 100 KB and the evaluation server must be able to compile it in less than 30 seconds. Submitted programs which do not meet these constraints will be rejected by the submission system and the contestant will be notified. There is a limit of 50 submissions per task.
A submission will be considered successful and accepted for grading only if the compilation was successful (no compile errors). Note that the submission will be accepted even if the program does not solve the sample tests correctly or within given resource constraints.
The Competition Server runs Linux, enforces memory and time limits, presents input data to the contestant's program, and captures the results. For tasks that require programs as solutions a contestant may run his or her active submission in the evaluation environment using the TEST interface. To use this facility, the contestant supplies an input file to the interface. The TEST interface will execute the program, enforcing the resource limitations for the particular task. The first 100 KB of screen (including error stream) output, the contents of the output data, execution time, and any applicable error messages will be displayed. A contestant can have at most one test execution in progress at a time.
The TEST interface allows the contestant to determine the exact behavior of his or her program in the evaluation environment with respect to compilation parameters, time and memory limits, and so on.
The TEST interface will be disabled during the last 30 minutes of each competition day.
Submitted input files must not exceed 2MB. Contestants may, however, compress input files using bzip2 (for example, using bzip2 -k input_file inside a shell). The test execution system will execute the program under Linux, enforcing the resource limitations for the particular task. The first 100 KB of the standard output, execution time, and any applicable error messages will be displayed.
Contestants may ask the support staff for assistance at any time. The staff members will not answer questions about the competition tasks, but will deliver Clarification Request Forms and printouts, help locate toilets and refreshments, and assist with computer and network problems. The only manner in which contestants are allowed to access the network is via web browser access to the designated Competition Server; even running a single 'ping' command is strictly prohibited. Contestants should never attempt to "fix" or "debug" or even "check" computer or network problems themselves; instead, they should ask for assistance.
Ending the Competition
Three warnings will be given at 15 minutes, 5 minutes, and 1 minute before the end of the round. Each warning will be given by an audible signal. The end of the round will be announced both verbally and by an audible signal. At the announcement ending the round, contestants must immediately stop working and wait at their desks without operating the computers or touching anything on their desks. An additional announcement will be made instructing them to leave their tables and exit the competition room.
At the end of the first competition round, any previously submitted items a contestant would like to use during the second competition round should be left at the workstation. All other items should be taken out of the competition hall, including task statements. At the end of the second competition round, contestants should remove all items including their mascots and dictionaries or any other previously submitted item; nothing should be left behind.
For output-only tasks, output files will be submitted one by one
and scored as described in the task description.
For tasks that require programs as solutions, the submitted source
files will be compiled under Linux, enforcing the source file size and
compilation time constraints. The following commands will be used to
compile solutions of tasks (say the task name is abc):
- For C:
gcc -o abc abc.c -std=c99 -O2 -DCONTEST -s -static -lm
- For C++:
g++ -o abc abc.cpp -O2 -DCONTEST -s -static
- For Pascal:
fpc -O1 -XS -dCONTEST abc.pas
Test run (batch and reactive tasks)
If the submitted solution compiled correctly, the grading system will, at some time later, execute the compiled program under Linux, enforcing the task-specific run-time and memory resource constraints with several inputs.
A test run refers to each time the solution is executed with a different input. For each test run the resource constraints will be enforced. If a program exceeds its allotment of any resource, the program is terminated and the test run is marked incorrect, without regard to any output produced.
If the program produces an output within the specified resource constraints, the output will be checked to verify its correctness. A score will be assigned to that specific test run depending on the output. Different test runs may have different maximum scores assigned to them.
If the test run is marked incorrect because it exceeded the resource constraints or because it produced an incorrect output it will be assigned a score of zero points.
For some tasks, partial credit may be awarded in a test run. Whenever this is possible it will be stated explicitly in the task description.
Upon termination, programs must always explicitly return an exit code 0. Other exit codes will be interpreted by the grading system as failures and no points will be awarded for the test run.
A test case is a set of one or more test runs. The score for a test case is defined as the minimum of the scores of its constituent test runs.
The purpose of a test case is to assess a well-defined characteristic of a program in a reliable and robust way. Such a characteristic could focus on, for instance, efficiency or special corner cases.
For every task there will be at least one "easy" test case, whose requirements will be separately stated in the task description.
The final score for any task will be obtained by adding the scores of each of the test cases for that task.
Only work which has been submitted to and accepted by the grading system will be awarded points.
Detailed feedback when submitting
For some of the tasks, detailed feedback may be enabled for 10 submissions. When detailed feedback is enabled for a task, every time a contestant submits a solution, he may choose to receive detailed feedback for that submission. If this option is selected, then after the submission is accepted, it will enter a queue for evaluation. A program can stay in the queue any time from a few seconds to a few minutes. When its turn comes the contestant's solution will be evaluated with some of the official test cases. After the evaluation is done, the Competition Server will show to the contestant a summary of their results on the executed test runs. The summary will contain the total number of test runs and the number of test runs for each of the following possible outcomes:
- The program executed within the constraints and the output produced by it was correct.
- The program executed within the constraints but the output produced by it was not correct.
- The program did not produce the output.
- The program exceeded the run-time limit specified by the task.
- The program terminated prematurely or irregularly (exit code other than 0, illegal instruction, floating point exception, invalid memory reference, etc).
No information on how the test runs are batched together or their assigned score will be given to the contestant. But for every test case if one of its test runs is enabled for feedback then all of its test runs will be executed.
No information on the actual data or the output produced by the contestant solution will be given to the contestant.
If detailed feedback is enabled for a task, after that number of submissions with feedback is exceeded the contestant may continue submitting solutions but no feedback will be given.
The detailed feedback will be disabled for all tasks during the last 30 minutes of each competition round.
Submitted programs are not allowed to:
- access the network,
- open and create files,
- attack the system security or the grader,
- execute other programs,
- change file system permissions,
- read file system information,
- make system calls not related to solving the competition task.
All of the above actions are considered cheating, and may result in disqualification.
Submitted solutions are judged using data which conforms to the specification given in the problem statement, but which is unknown to competitors during the competition.
Provisional grades, based on these tests, are delivered to team leaders' mailboxes as soon as they are available, shortly after competition ends. The judge data are made available electronically in the translation environments. Contestants and team leaders may use the translation workstations to verify that the grades are assessed correctly.
A Team Leader may file an appeal by completing an Appeal Form, available near the mailboxes, and submitting the completed form to the mailbox labeled "Appeals" at most 6 hours after results have been published. Every appeal will be reviewed by the Scientific Committee and the team leader will be notified the committee's decision. All appeals and their disposition will be summarized at the final GA meeting of that competition day.
In the event that a mistake is discovered in the grading of a task, every successful submission of that task will be re-graded and re-scored whether or not the scoring of that submission has been appealed. Note that re-scoring may result in a higher or lower score for any contestant. Should anyone's score change after grading results have been distributed, new results will be printed and distributed to them.
Code of Conduct
- interfering with other contestants' activities,
- damaging the competition or evaluation facilities,
- accessing the network or any computer other then their own during a competition round by any means other than pointing a web browser at the Competition Server as directed,
- subverting the security of any computers (e.g., making any material accessible to the network from their computer),
- interfering with the execution of the competition in any way,
- communicating in any way with anyone other than the competition staff during a competition round (whether providing or receiving information or materials),
- or attempting to do any of the above
will be considered to be cheating and is subject to disqualification. Also refer to the list above of prohibited program actions which are considered cheating and may result in disqualification.
Cheating will be taken very seriously and rules will be strictly enforced. Any contestant who is cheating in any way will be immediately disqualified, at the discretion of the International Scientific Committee. It is of the utmost importance that team leaders speak to their contestants about cheating, instructing and urging all contestants not to cheat, explaining the relevant rules and verifying that all contestants fully understand them.