========================== Frequently Asked Questions ========================== General Questions ================= What is Django Evolution? ------------------------- When you run ``./manage.py syncdb``, Django will look for any new models that have been defined, and add a database table to represent those new models. However, if you make a change to an existing model, ``./manage.py syncdb`` will not make any changes to the database. This is where **Django Evolution** fits in. Django Evolution is an extension to Django that allows you to track changes in your models over time, and to update the database to reflect those changes. When will Django Evolution be merged into Django trunk? ------------------------------------------------------- Possibly never, and certainly not before it's ready. Django Evolution is still in the early stages of development; considering plans for merging to the Django trunk would be premature. Installation ============ How do I install Django Evolution? ---------------------------------- There have not been any official releases of Django Evolution. If you want to use Django Evolution, you will need to use a subversion checkout. Check out the Django Evolution sources, and put the checkout directory into your Python Path. Then add 'django_evolution' to the list of installed applications for the project in which you want to use Django Evolution. What version of Django does Django Evolution support? ----------------------------------------------------- Django Evolution requires that you are using Django v1.0. You should also be able to use any SVN checkout from revision 8296 or higher. You cannot use Django Evolution with a 0.96 (or earlier) release of Django. Management ========== Who is behind Django Evolution? ------------------------------- Django Evolution was originally developed as a part time project by two guys based in Perth, Western Australia: `Ben Khoo`_ and `Russell Keith-Magee`_. Russell is also a core developer of Django itself. Russell is now an emeritus contributor. the project is maintained by `Christian Hammond`_. .. _`Ben Khoo`: mailto:khoobks@westnet.com.au .. _`Russell Keith-Magee`: mailto:russell@keith-magee.com .. _`Christian Hammond`: mailto:chipx86@chipx86.com Where should I ask questions about Django Evolution? ---------------------------------------------------- Django Evolution has a `Google Groups mailing list`_ for discussing the usage and development of the project. .. _`Google Groups mailing list`: http://groups.google.com/group/django-evolution There is also a read-only mailing list if you wish to be notified whenever code is checked into the Subversion repository, or whenever a ticket is added or modified. .. _`read-only mailing list`: http://groups.google.com/group/django-evolution-changes I think I've found a bug! What should I do? ------------------------------------------- Django Evolution has a `bug tracker`_. Check the list of bugs that are already known; if your bug doesn't appear to be logged, please add it to the list. When you lodge your bug, *please* remember to provide clear instructions on how to reproduce your problem. This means providing sample models, sample migrations, and any error logs that appeared during the evolution process. .. _`bug tracker`: http://code.google.com/p/django-evolution/issues/list Status ====== What works? ----------- The general framework should be stable. Any difficulties installing Django Evolution into your project, running the evolve command, or identifying changes in your models should be reported as a bug. The following is a list of the model changes that we believe to be working: * Addition of simple fields (e.g., a new CharField). * Addition of Many To Many Fields. * Deletion of most fields (of any type). * Deletion of models. * Deletion of entire applications. * Renaming of a field. Renames won't ever appear in a hinted evolution, but they can be manually added as a replacement for the Add+Delete pair that a hinted evolution will provide. Primary development for Django Evolution has used the Psycopg1 interface to PostgreSQL. As a result, this is the most stable database interface. A Psycopg2 PostgreSQL backend also exists, and should work without difficulty. The SQLite backend is also fairly reliable. The MySQL backend works for most mutations, with some exceptions (details below). What doesn't work? ------------------ Django Evolution is a work in progress, and as a result, there are things that don't work, or don't work as expected. The following model changes are known to cause difficulty: * Addition of ForeignKey fields. A field will be added, but the foreign key constraints are not correctly constructed. * Deletion of primary keys (including adding a manual primary key as a replacement for an automatically installed primary key) * Deletion of fields referenced by other database constraints. Dependencies between applications can also cause difficulties. If you have changes in one application that are dependent on changes in another application, you may experience difficulties in applying evolutions. Django Evolution is known to work well for PostgreSQL and SQLite. This is because the core developers both use PostgreSQL and SQLite on a daily basis. Support for MySQL is less complete. The following features are known to be buggy or broken under MySQL: * Renaming of primary key fields and foreign key fields. Improving MySQL support (as well as adding support for other database backends) is one big area where contributions would be most welcome. Usage ===== How do I remove an entire application from my project? ------------------------------------------------------ If you remove an application from the ``INSTALLED_APPS`` list, Django Evolution will not attempt to remove the tables for this application. If you want to remove any tables associated with a deleted application, you must specify the ``--purge`` option to evolve. Why does Django Evolution generate an error when hinting an evolution? ---------------------------------------------------------------------- Django Evolution is a work in progress. The infrastructure necessary to identify model changes is in place and is (to the best of our knowledge) complete. However, the definitions required to handle every possible type of model change have not yet been developed. As a result, there are some types of change that you can make to a model for which Django Evolution cannot produce a corresponding SQL mutation. We hope to eventually support these changes with a hinted mutation. However, in the interim, the workaround is to manually define an SQL mutation and include it as a stored evolution in your project. My OneToOneField has changed as a result of the QuerySet Refactor merge! What should I do? ------------------------------------------------------------------------------------------ First - some background. `Django revision 7477`_ saw the merging of the `Queryset Refactor branch`_ back into the Django trunk. This change introduced a lot of behind-the-scenes improvements to the way queries are composed. However, this also introduced some subtle `backwards-incompatible changes`_ that can affect your database definitions. In turn, these changes have an effect on the way Django Evolution treats your database. For example, consider the case of the following model defining a user profile:: class UserProfile(models.Model): user = models.OneToOneField() foobar = models.CharField(max_length=40) If you were to run syncdb on this model prior to Django revision 7477, you would get a database definition that looks something like:: CREATE TABLE "test_app_userprofile" ( "user_id" integer NOT NULL PRIMARY KEY REFERENCES "auth_user" ("id") DEFERRABLE INITIALLY DEFERRED, "foobar" varchar(40) NOT NULL ); However, if you were to run syncdb using a version of Django after revision 7477, you would get the following:: CREATE TABLE "test_app_userprofile" ( "id" serial NOT NULL PRIMARY KEY, "user_id" integer NOT NULL UNIQUE REFERENCES "auth_user" ("id") DEFERRABLE INITIALLY DEFERRED, "foobar" varchar(40) NOT NULL ); The suggested response to this change is to add "primary_key=True" to your OneToOneField defintion. While this will result in the same functionality at runtime, it will be identified by Django Evolution as a change in model signature - even though no database change is required. What is required is a way to ignore the change of signature. This can be achieved using an empty SQL mutation that provides a signature update function. To update the previous example, put the following into your mutation sequence definition:: def qsrf_update(app_label, proj_sig): app_sig = proj_sig[app_label] model_sig = app_sig['UserProfile'] model_sig['fields']['user'] = { 'primary_key': True } SEQUENCE = [ ... SQLMutation('qsrf-update', [], qsrf_update) ... ] This defines a signature update function called "qsrf_update()" that puts a "primary_key" definition into the signature for the OneToOneField. This function is used by an SQL mutation called 'qsrf-update' that contains no actual SQL statements. So, when this mutation is applied, the signature will be updated, but no changes will be made to the database. .. _`Django revision 7477`: http://code.djangoproject.com/changeset/7477 .. _`Queryset Refactor branch`: http://code.djangoproject.com/wiki/QuerysetRefactorBranch .. _`backwards-incompatible changes`: http://code.djangoproject.com/wiki/QuerysetRefactorBranch#Backwardsincompatiblechanges Design ====== What are the design goals for Django Evolution? ----------------------------------------------- Django Evolution has the following design goals: 1. Provide a way for simple changes to be applied to a database without significant user intervention. 2. Provide an entry point for using raw SQL as a migration tactic. 3. Provide a way to 'hint and tweak' - i.e., use an automatic method to generate a first pass at a migration, but provide a way for a developer to fine tune the migration before application. 4. Provide a way to be self-documenting and self-auditing - i.e., keep a permanent archive of the changes that have been made, which ones have been applied to which database, etc. 5. Provide some manner of validation that a migration, when applied, will be correct. 6. Provide an easy way for end users to integrate their own custom migration needs into the framework. Isn't this just ActiveRecord::Migration in Python? -------------------------------------------------- Superficially, yes, but not really. There is a degree of similarity - the Django Evolution syntax is a DSL for describing the changes that have been made to a model. However, there are also some significant differences. In addition to ActiveRecord::Migration, Django Evolution provides: 1. An audit trail - a permanent archive in the database of the changes that have been applied, and when. 2. An automated hinting scheme. If you don't want to write the migrations yourself, you don't have to. 3. Validation that an evolution script will leave the database in a state consistent with the current model definition. These differences are largely afforded by the model-centric design of Django itself. Whereas a Ruby on Rails model is a description of a database that has been created by hand, Django uses the Python model to creates the database. As a result, the model definition is canonical - not the database. This means that audit, hinting, and verification schemes can use the Django model as a point of reference. A Django Evolution script is much more than just an alternate syntax for SQL ALTER statements - it is an evolution scheme that is bound to the canonical model definition. .. admonition:: Ultracrepidation We're not Rails developers, so when we talk about Ruby on Rails and ActiveRecord::Migration, it's not from extensive personal experience. If we have misrepresented or misunderstood Ruby on Rails or ActiveRecord::Migration, feel free to correct us. Why do you generate signatures based on the model, rather than introspecting the database? ------------------------------------------------------------------------------------------ There are two arguments for using model-based signatures rather than database introspection: One is practical; the other is philosophical. The practical argument is that introspection is _much_ harder to do properly. To prove the point: ``inspectdb`` has been in Django since the beginning, and it still doesn't work right 100% of the time. Introspection of fields with custom column names or custom field types will always be very difficult to resolve. By using model-based signatures, we always have access to all the properties that Django considers significant, in a format directly relevant to Django - as a result, it was possible to get model-based signatures working with pretty much 100% accuracy in relatively short order. The philosophical argument is that by making the model the canonical reference point (rather than the database) it puts the emphasis on Django as the centre of the process, and discourages external modification. This has the side effect of encouraging users to store (which is to say *document*) all the changes they make to the database, rather than just poking the schema as they see fit. This also has the advantage of being consistent with general Django philosophy - for example, in default usage, you write a Django model, and Django creates the tables. You *can* manually create the tables if you want, and you *can* manually tweak the tables if you need to - but if you do, Django doesn't guarantee that that anything will work as expected. Django provides initial SQL hooks that implicitly encourage you to store/document any external modifications. Django doesn't go out of its way to make life difficult in this regard, but the core design of Django isn't going to change fundamentally in order to accommodate externalities. Contributing code ================= How can I get started contributing code to Django Evolution? ------------------------------------------------------------ Thanks for asking! If you're interested in contributing to Django Evolution, please send a private email to Christian. However, be advised that if you you want your request to be taken seriously, you need to have a track record of producing high quality patches. A good place to start would be to pick a non-trivial missing feature from the ticket database, and implement a fix. I submitted a bug fix in the ticket system several weeks ago. Why are you ignoring my patch? -------------------------------------------------------------------------------------------- Don't worry: We're not ignoring you! It's important to understand there is a difference between "a ticket is being ignored" and "a ticket has not been attended to yet." Django Evolution is a volunteer project, and the core developers may sometimes get distracted by other things; if this happens, a ticket may sit unattended for a while. Don't worry - we'll get to your issue eventually. Besides, if your feature request stands no chance of inclusion in Django Evolution, we won't ignore it -- we'll just close the ticket. So if your ticket is still open, it doesn't mean we're ignoring you; it just means we haven't had time to look at it yet.