迁移 Django Model id 为 uuid

https://www.klette.us/migrate-django-models-to-uuid-primary-key/

又是一篇渣翻

老设计决策有时候会让你很难受,下面就是一个例子。

我手上的一个 Django Model 是这样的:

<span class="token keyword">class</span> <span class="token class-name">Municipality</span><span class="token punctuation">(</span>models<span class="token punctuation">.</span>Model<span class="token punctuation">)</span><span class="token punctuation">:</span>
    code <span class="token operator">=</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">2</span><span class="token punctuation">,</span> primary_key<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">)</span>
    name <span class="token operator">=</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">100</span><span class="token punctuation">)</span>

此外,还有一个外键指向它

<span class="token keyword">class</span> <span class="token class-name">ZipCode</span><span class="token punctuation">(</span>models<span class="token punctuation">.</span>Model<span class="token punctuation">)</span><span class="token punctuation">:</span>
    code <span class="token operator">=</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">2</span><span class="token punctuation">,</span> primary_key<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">)</span>
    municipality <span class="token operator">=</span> models<span class="token punctuation">.</span>ForeignKey<span class="token punctuation">(</span>Municipality<span class="token punctuation">)</span>

现在,我们需要让 municipality 支持多个国家,显然一个唯一 field 将会引起冲突,不够用。(译者:这个理由有点牵强)

对于所有代码的现代部分,我们使用UUID作为主键。所以我们想要 migrate municaipality的主键到UUID,并且维持了原有的关系。(译者:外键和自增主键都是数据库性能的杀手。)

2017年9月份,Django 依然不支持优雅的迁移主键,所以我们自己做了(译者:我也不知道现在行不行。)

我们尝试了许多 magic 解决方案,但是我们在 migrations 系统这个被难住了,并且不能检测以及很好的处理改变。

在经过一小段研究和错误,我们找到了以下的解决方案。尽管这还有一些小问题,但是的确有效。

再一次提醒,从数据库的角度,当你定义了一个ForeignKey field 在 Django 中,Django将会创建一个数据库列,是同样的类型,作为 referenced model 的主键,并且增加外键约束。所以在上面的例子中,我们有两个表:

<span class="token keyword">CREATE</span> <span class="token keyword">TABLE</span> municipality <span class="token punctuation">(</span>
   code <span class="token keyword">varchar</span><span class="token punctuation">(</span><span class="token number">2</span><span class="token punctuation">)</span> <span class="token keyword">PRIMARY</span> <span class="token keyword">KEY</span> <span class="token operator">NOT</span> <span class="token boolean">NULL</span><span class="token punctuation">,</span>
   name <span class="token keyword">varchar</span><span class="token punctuation">(</span><span class="token number">100</span><span class="token punctuation">)</span>
<span class="token punctuation">)</span><span class="token punctuation">;</span>

<span class="token keyword">CREATE</span> <span class="token keyword">TABLE</span> zipcode <span class="token punctuation">(</span>
   code <span class="token keyword">varchar</span><span class="token punctuation">(</span><span class="token number">2</span><span class="token punctuation">)</span> <span class="token keyword">PRIMARY</span> <span class="token keyword">KEY</span> <span class="token operator">NOT</span> <span class="token boolean">NULL</span><span class="token punctuation">,</span>
   municipality_id <span class="token keyword">VARCHAR</span><span class="token punctuation">(</span><span class="token number">2</span><span class="token punctuation">)</span> <span class="token keyword">REFERENCES</span><span class="token punctuation">(</span>municipality<span class="token punctuation">.</span>id<span class="token punctuation">)</span> <span class="token operator">NOT</span> <span class="token boolean">NULL</span>
<span class="token punctuation">)</span><span class="token punctuation">;</span>

所以我们需要解除外键约束,更换 root model,然后映射新的主键到旧的上,并且重新应用外键到上面去。

我们首先打破外键

<span class="token keyword">class</span> <span class="token class-name">ZipCode</span><span class="token punctuation">(</span>models<span class="token punctuation">.</span>Model<span class="token punctuation">)</span><span class="token punctuation">:</span>
    code <span class="token operator">=</span> <span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span>  <span class="token comment" spellcheck="true"># Same as before</span>
    municipality <span class="token operator">=</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">2</span><span class="token punctuation">)</span>
python manage.py makemigrations -n break_zipcode_muni_foreignkey

现在,·Municipality·modle 没有被任何外键所 refer,我们可以在上面进行工作了。

增加一个新的 id field:

<span class="token keyword">class</span> <span class="token class-name">Municipality</span><span class="token punctuation">(</span>models<span class="token punctuation">.</span>Model<span class="token punctuation">)</span><span class="token punctuation">:</span>
    id <span class="token operator">=</span> models<span class="token punctuation">.</span>UUIDField<span class="token punctuation">(</span>default<span class="token operator">=</span>uuid<span class="token punctuation">.</span>uuid4<span class="token punctuation">)</span>
python manage.py makemigrations -n add_id_field_to_muni

处于某些原因,默认值在我的案例中不 work,所以我增加了一个过程到创建的 migration,来创建新的唯一 id。

<span class="token keyword">def</span> <span class="token function">create_ids</span><span class="token punctuation">(</span>apps<span class="token punctuation">,</span> schema_editor<span class="token punctuation">)</span><span class="token punctuation">:</span>
    Municipality <span class="token operator">=</span> apps<span class="token punctuation">.</span>get_model<span class="token punctuation">(</span><span class="token string">'loc'</span><span class="token punctuation">,</span> <span class="token string">'Municipality'</span><span class="token punctuation">)</span>
    <span class="token keyword">for</span> m <span class="token keyword">in</span> municipality<span class="token punctuation">:</span>
        m<span class="token punctuation">.</span>id <span class="token operator">=</span> uuid<span class="token punctuation">.</span>uuid4<span class="token punctuation">(</span><span class="token punctuation">)</span>
        m<span class="token punctuation">.</span>save<span class="token punctuation">(</span><span class="token punctuation">)</span>
        
<span class="token comment" spellcheck="true"># ...</span>

operations <span class="token operator">=</span> <span class="token punctuation">[</span>
    migrations<span class="token punctuation">.</span>AddField<span class="token punctuation">(</span><span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">)</span><span class="token punctuation">,</span>
    migrations<span class="token punctuation">.</span>RunPython<span class="token punctuation">(</span>code<span class="token operator">=</span>create_ids<span class="token punctuation">)</span><span class="token punctuation">,</span>
<span class="token punctuation">]</span>

现在我们有一个UUID ·id·field 在·Municipality 中,然后我们应该可以更换主键了。

<span class="token keyword">class</span> <span class="token class-name">Municipality</span><span class="token punctuation">(</span>models<span class="token punctuation">.</span>Model<span class="token punctuation">)</span><span class="token punctuation">:</span>
    id <span class="token operator">=</span> models<span class="token punctuation">.</span>UUIDField<span class="token punctuation">(</span>default<span class="token operator">=</span>uuid<span class="token punctuation">.</span>uuid4<span class="token punctuation">,</span> primary_key<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">)</span>
    code <span class="token operator">=</span> models<span class="token punctuation">.</span>CharField<span class="token punctuation">(</span>max_length<span class="token operator">=</span><span class="token number">2</span><span class="token punctuation">,</span> unique<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">)</span>

创建 migration,并且确保在code上的AlterField操作在id之前。我们已经在id上增加了primary_key并且增加了 unique=True 到code field。constraint 没了,在我们删除 primary_key 的时候就没了。

让我们开始一个空的迁移

python manage.py makemigrations --empty -n fix_zipcode_fk_to_muni_uuid loc

打开文件,输入

<span class="token keyword">def</span> <span class="token function">match</span><span class="token punctuation">(</span>apps<span class="token punctuation">,</span> schema_editor<span class="token punctuation">)</span><span class="token punctuation">:</span>
    ZipCode <span class="token operator">=</span> apps<span class="token punctuation">.</span>get_model<span class="token punctuation">(</span><span class="token string">'loc'</span><span class="token punctuation">,</span> <span class="token string">'ZipCode'</span><span class="token punctuation">)</span>
    Muni <span class="token operator">=</span> apps<span class="token punctuation">.</span>get_model<span class="token punctuation">(</span><span class="token string">'loc'</span><span class="token punctuation">,</span> <span class="token string">'Municipality'</span><span class="token punctuation">)</span>
    <span class="token keyword">for</span> zip_code <span class="token keyword">in</span> ZipCode<span class="token punctuation">.</span>object<span class="token punctuation">.</span>all<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        zip_code<span class="token punctuation">.</span>temp_muni <span class="token operator">=</span> Muni<span class="token punctuation">.</span>get<span class="token punctuation">(</span>code<span class="token operator">=</span>z<span class="token punctuation">.</span>municipality<span class="token punctuation">)</span>
        zip_code<span class="token punctuation">.</span>save<span class="token punctuation">(</span><span class="token punctuation">)</span>
    
<span class="token comment" spellcheck="true"># ...</span>
operations <span class="token operator">=</span> <span class="token punctuation">[</span>
    migrations<span class="token punctuation">.</span>AddField<span class="token punctuation">(</span>
        model_name<span class="token operator">=</span><span class="token string">'zipcode'</span><span class="token punctuation">,</span>
        name<span class="token operator">=</span><span class="token string">'temp_muni'</span><span class="token punctuation">,</span>
        field<span class="token operator">=</span>models<span class="token punctuation">.</span>UUIDField<span class="token punctuation">(</span>null<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">)</span><span class="token punctuation">,</span>
    <span class="token punctuation">)</span><span class="token punctuation">,</span>
    migrations<span class="token punctuation">.</span>RunPython<span class="token punctuation">(</span>code<span class="token operator">=</span>match<span class="token punctuation">)</span><span class="token punctuation">,</span>
    migrations<span class="token punctuation">.</span>RemoveField<span class="token punctuation">(</span>model_name<span class="token operator">=</span><span class="token string">'zipcode'</span><span class="token punctuation">,</span> name<span class="token operator">=</span><span class="token string">'municipality'</span><span class="token punctuation">)</span><span class="token punctuation">,</span>
    migrations<span class="token punctuation">.</span>RenameField<span class="token punctuation">(</span>
        model_name<span class="token operator">=</span><span class="token string">'zipcode'</span><span class="token punctuation">,</span> old_name<span class="token operator">=</span><span class="token string">'temp_muni'</span><span class="token punctuation">,</span> new_name<span class="token operator">=</span><span class="token string">'municipality'</span><span class="token punctuation">)</span><span class="token punctuation">,</span>
    migrations<span class="token punctuation">.</span>AlterField<span class="token punctuation">(</span>
        model_name<span class="token operator">=</span><span class="token string">'zipcode'</span><span class="token punctuation">,</span>
        name<span class="token operator">=</span><span class="token string">'municipality'</span><span class="token punctuation">,</span>
        field<span class="token operator">=</span>models<span class="token punctuation">.</span>ForeignKey<span class="token punctuation">(</span>
            on_delete<span class="token operator">=</span>django<span class="token punctuation">.</span>db<span class="token punctuation">.</span>models<span class="token punctuation">.</span>deletion<span class="token punctuation">.</span>PROTECT<span class="token punctuation">,</span>
            to<span class="token operator">=</span><span class="token string">'municipality'</span><span class="token punctuation">)</span>
<span class="token punctuation">]</span>
  1. 增加一个临时的field来保存 Municipality 的UUID,我们不让他是一个ForeignKeyfield,否则 Django confuse。
  2. 我们运行 match 函数来寻找新的 id,通过寻找旧的 key,并且存储到临时的 field。
  3. 删除旧的 municipality field
  4. 重命名临时 field 到 municipality
  5. 最后迁移到一个外键,然后创建我们需要 constraint
译者:这样改最大的好处就是改了 UUID,但是没有破坏之前的逻辑,可以说是谨慎的迁移吧。不过线上的数据既然在跑着,就这么迁移了,会不会出问题呢。

下面还有一些内容。自从我们将 migrations 分离成多个文件,我们让代码变得很脆弱(如果后面的某些 migrations 失败了)。这将会让我们的应用进入一个 unworkable 的状态。所以确保测试一下migrations。你可通过手工把这所有的步骤整合到一个migration里,但是如果你有来自多个不同app的 reference,你可能需要把这些步骤再分开。

logging

迁移的过程中,你可能遇到很多问题,所以一个比较不错的方式,是创建一个简单的迁移 logging。

<span class="token keyword">def</span> <span class="token function">log</span><span class="token punctuation">(</span>message<span class="token punctuation">)</span><span class="token punctuation">:</span>
    <span class="token keyword">def</span> <span class="token function">fake_op</span><span class="token punctuation">(</span>apps<span class="token punctuation">,</span> schema_editor<span class="token punctuation">)</span><span class="token punctuation">:</span>
        <span class="token keyword">print</span><span class="token punctuation">(</span>message<span class="token punctuation">)</span>
    <span class="token keyword">return</span> fake_op
    
    
 <span class="token comment" spellcheck="true"># ...</span>
 
 operations <span class="token operator">=</span> <span class="token punctuation">[</span>
     migration<span class="token punctuation">.</span>RunPython<span class="token punctuation">(</span>log<span class="token punctuation">(</span><span class="token string">'Step 1'</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">,</span>
     migration<span class="token punctuation">.</span>AlterField<span class="token punctuation">(</span><span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">)</span><span class="token punctuation">,</span>
     migration<span class="token punctuation">.</span>RunPython<span class="token punctuation">(</span>log<span class="token punctuation">(</span><span class="token string">'Step 2'</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">,</span>
     <span class="token comment" spellcheck="true"># ...</span>
 <span class="token punctuation">]</span>

想要观察 Django 运行了那些 SQL 语句,运行python manage.py sqlmigrate <appname><migration_number>,这是一个超级有用的方法。

 

Contents

comments powered by Disqus