Note: This unit version has not been officially published yet and is subject to change!
CSYS5030: Information Theory and SelfOrganisation (2019  Semester 2)
Unit:  CSYS5030: Information Theory and SelfOrganisation (6 CP) 
Mode:  NormalDay 
On Offer:  Yes 
Level:  Postgraduate 
Faculty/School:  Faculty of Engineering and Information Technologies 
Unit Coordinator/s: 
Dr Lizier, Joseph

Session options:  Semester 2 
Versions for this Unit: 
Campus:  Camperdown/Darlington 
PreRequisites:  None. 
Brief Handbook Description:  The dynamics of complex systems are often described in terms of how they process information and selforganise; for example regarding how genes store and utilise information, how information is transferred between neurons in undertaking cognitive tasks, and how swarms process information in order to collectively change direction in response to predators. The language of information also underpins many of the central concepts of complex adaptive systems, including order and randomness, selforganisation and emergence. Shannon information theory, which was originally founded to solve problems of data compression and communication, has found contemporary application in how to formalise such notions of information in the world around us and how these notions can be used to understand and guide the dynamics of complex systems. This unit of study introduces information theory in this context of analysis of complex systems, foregrounding empirical analysis using modern software toolkits, and applications in timeseries analysis, nonlinear dynamical systems and data science. Students will be introduced to the fundamental measures of entropy and mutual information, as well as dynamical measures for time series analysis and information flow such as transfer entropy, building to higherlevel applications such as feature selection in machine learning and network inference. They will gain experience in empirical analysis of complex systems using comprehensive software toolkits, and learn to construct their own analyses to dissect and design the dynamics of selforganisation in applications such as neural imaging analysis, natural and robotic swarm behaviour, characterisation of risk factors for and diagnosis of diseases, and financial market dynamics. 
Assumed Knowledge:  Competency in 1st year mathematics, and basic computer programming skills are assumed. Competency in 1st year undergraduate level statistics (for example, covering probabilities, conditional probabilities, Gaussian distribution, correlations, statistical significance/hypothesis testing and pvalues). An exposure to linear algebra would be useful but not mandatory. 
Lecturer/s: 
Dr Lizier, Joseph


Timetable:  CSYS5030 Timetable  
Time Commitment: 


T&L Activities:  This unit of study comprises of regular lectures, as well as laboratory / tutorial sessions. These sessions will take place in a lab with access to relevant computer facilities. Depending on the syllabus, some weeks will comprise tutorials where students will solve problems with the help of tutors, and other weeks will comprise programming or software based laboratory experiments. During lab sessions with a programming/software component, tutors will be present to assist students develop relevant programming or other computing skills. Students will use independent study time to further develop their computing skills and to practise solving analytical problems. 
Attributes listed here represent the key course goals (see Course Map tab) designated for this unit. The list below describes how these attributes are developed through practice in the unit. See Learning Outcomes and Assessment tabs for details of how these attributes are assessed.
Attribute Development Method  Attribute Developed 
Develop through lectures, case studies and tutorial examples, a working knowledge of techniques and tools that can be used for complex systems design.  Design (Level 4) 
Develop through lectures and tutorial discussions an understanding of relevant concepts related to information theory, and how to apply these techniques to understand complex systems.  Engineering/IT Specialisation (Level 5) 
Develop through lectures and tutorial discussions an understanding of: 1  Programming skills needed for using information theory 2  Fluency in using open source tools and toolkits for information theory related calculations 3  How to apply information theory calculations empirically using software to realworld data sets 
Maths/Science Methods and Tools (Level 5) 
Develop through assignments and case studies an ability to critically dissect and understand the structure and function of complex systems, and the ability to describe this understanding efficiently and quantitatively. Be able to effectively use information theory in understanding the dynamics of complex systems. 
Information Seeking (Level 3) 
For explanation of attributes and levels see Engineering & IT Graduate Outcomes Table 2018.
Learning outcomes are the key abilities and knowledge that will be assessed in this unit. They are listed according to the course goal supported by each. See Assessment Tab for details how each outcome is assessed.
Information Seeking (Level 3)Assessment Methods: 


Assessment Description: 
The assessments will consist of four assignments. Assignments 14 will be completed individually. Literature review: You will write a short appraisal of an article / report / blog post which provides an informationtheoretic analysis of empirical data, and critically evaluate that study. The reviews will be socially shared and discussed in class. Information theory exercises: You will demonstrate your understanding of the informationtheoretic concepts taught in class in calculation exercises (involving both involve mathematical and computational tasks) and short answer questions. Info theory project presentation and report: You will create an informationtheoretic analysis of a data set of your choosing, selecting appropriate tools to answer a question of interest regarding relationships in that data. You will deliver an inclass presentation, and write a report, describing your approach, the results, discussing implications for the system under study, and critically evaluating your findings. Written assignments and exercises (without formally applying for special consideration) will be assessed a penalty of 7% per day until reaching the day limit where special consideration is required. Extensions for the Project Presentation will only be granted in the case where formal Special Consideration has been applied for and approved. Then university has authorised and mandated the use of text based similarity detecting software TURNITIN for all text based written assignments. IMPORTANT: There may be statistically defensible moderation when combining the marks from each component to ensure consistency of marking between markers, and alignment of final grades with unit outcomes and grade descriptors. * means this assessment must be repeated or will be replaced with a different assessment if missed due to special consideration 

Assessment Feedback:  Feedback for assignments will be through Canvas/TurnItIn elearning portal, though which these assignments will be submitted.  
Grading: 


Policies & Procedures:  See the policies page of the faculty website at http://sydney.edu.au/engineering/studentpolicies/ for information regarding university policies and local provisions and procedures within the Faculty of Engineering and Information Technologies. 
Recommended Reference/s: 
Note: References are provided for guidance purposes only. Students are advised to consult these books in the university library. Purchase is not required.

Note that the "Weeks" referred to in this Schedule are those of the official university semester calendar https://web.timetable.usyd.edu.au/calendar.jsp
Week  Description 
Week 1  Lecture/Tutorial: Introduction to information theory and entropy 
Week 2  Lecture/Tutorial: Entropy and mutual information I 
Week 3  Lecture/Tutorial: Entropy and mutual information II 
Week 4  Lecture/Tutorial: Informationtheoretic estimators and the JIDT toolkit 
Assessment Due: Literature review*  
Week 5  Lecture/Tutorial: Selforganisation and case studies 
Week 6  Lecture/Tutorial: Statistical significance 
Week 7  Lecture/Tutorial: Information dynamics (timeseries analysis) I 
Assessment Due: Information theory exercises*  
Week 8  Lecture/Tutorial: Information dynamics (timeseries analysis) II 
Week 9  Lecture/Tutorial: Information dynamics (timeseries analysis) III 
Week 10  Lecture/Tutorial: Network inference 
Week 11  Lecture/Tutorial: Project work 
Week 12  Lecture/Tutorial: Project presentations 
Assessment Due: Project presentation*  
Week 13  Lecture/Tutorial: Wrap up / Project work 
STUVAC (Week 14)  Assessment Due: Project report* 
Course Relations
The following is a list of courses which have added this Unit to their structure.
Course  Year(s) Offered 
Graduate Diploma in Complex Systems  2017, 2018, 2019 
Master of Complex Systems  2017, 2018, 2019 
Course Goals
This unit contributes to the achievement of the following course goals:
Attribute  Practiced  Assessed 
Information Seeking (Level 3)  Yes  23% 
Maths/Science Methods and Tools (Level 5)  Yes  45.25% 
Engineering/IT Specialisation (Level 5)  Yes  19.5% 
Design (Level 4)  Yes  12.25% 
These goals are selected from Engineering & IT Graduate Outcomes Table 2018 which defines overall goals for courses where this unit is primarily offered. See Engineering & IT Graduate Outcomes Table 2018 for details of the attributes and levels to be developed in the course as a whole. Percentage figures alongside each course goal provide a rough indication of their relative weighting in assessment for this unit. Note that not all goals are necessarily part of assessment. Some may be more about practice activity. See Learning outcomes for details of what is assessed in relation to each goal and Assessment for details of how the outcome is assessed. See Attributes for details of practice provided for each goal.