
# Introduction
At Empathy Works, we specialize in creating tailored solutions that drive efficiency and innovation for our clients. One of our recent projects involved helping a client rebuild their entire web application. The client had a monolithic application responsible for every aspect of their business, including the front-end CMS, sales, user management, training, project management, and orders. This setup made it difficult for them to iterate quickly and make changes efficiently.

# The Challenge
The monolithic nature of the client's application presented several challenges:

1. **Slow Iteration**: Making changes to the marketing website or front-end required significant developer involvement, slowing down the process.
2. **Complexity**: The intertwined nature of the application components made it difficult to isolate issues or update specific parts without affecting the entire system.
3. **Scalability**: The client needed a more flexible solution to scale different parts of their application independently.

# The Solution
Empathy Works devised a strategy to split the monolithic application into two distinct, optimized components: the front-end website and the e-commerce system. Our approach included:

1. **Front-End Website with WordPress**: We used WordPress to handle the front-end CMS, allowing the client to easily update content, experiment, and leverage the extensive WordPress ecosystem. This separation enabled the client to move quickly on building and updating their website.
2. **Custom Django Application for E-Commerce**: We developed a custom Django application to manage user accounts, orders, e-commerce, and the authentication flow. This application was tailored to meet the specific needs of the client’s business, providing a robust and flexible backend solution.
3. **Deep Integration**: Despite the technical separation, we ensured a seamless user experience by deeply integrating WordPress and the Django application. This integration allowed end users to navigate smoothly between the marketing website and the e-commerce system without noticing any difference.

# The Outcome
The new system provided significant benefits:

- **Faster Iteration**: The client could now make changes to their front-end website quickly and independently, allowing for more experimentation and innovation.
- **Enhanced Flexibility**: By leveraging WordPress for the front-end, the client could involve additional designers and developers familiar with the WordPress ecosystem.
- **Custom Solutions**: The custom Django application handled the unique requirements of the client’s e-commerce and user management system, offering a tailored and optimized solution.
- **Scalability**: The separation of concerns allowed each component to scale independently, improving the overall efficiency and performance of the system.

# Conclusion
By transforming a monolithic application into two specialized, integrated systems, Empathy Works enabled our client to achieve greater flexibility, faster iteration, and enhanced performance. This case study highlights our commitment to leveraging off-the-shelf tools and custom development to deliver the best solutions for our clients’ unique needs.

Breaking the Monolith: Crafting Two Streamlined, Optimized Systems


# Introduction
In today's data-driven world, the ability to access and analyze data efficiently is crucial for businesses aiming to make informed decisions. At Empathy Works, we specialize in empowering our clients with the tools they need to harness their data effectively. One of our recent projects involved working with a client to implement Metabase, a powerful business intelligence tool, to unlock access to their data and make it available to all team members, regardless of their technical expertise.

# The Challenge
Our client faced a significant challenge: they had vast amounts of data spread across various applications, but accessing and analyzing this data was limited to a small group of data analysts. The goal was to democratize data access, allowing everyone on the team, from analysts to curious non-technical staff, to explore and utilize the data. The key requirements were ensuring that the data could be accessed safely and securely, with robust permissions in place to protect sensitive information.

# The Solution
Empathy Works collaborated closely with the client to implement Metabase, a user-friendly, open-source business intelligence tool. Metabase was chosen for its ability to provide an intuitive interface for exploring data and its strong support for permissions and security. Our approach included the following steps:

- **Data Integration**: We connected Metabase to the client’s various data sources, ensuring that all relevant data was aggregated and accessible from a single platform.
- **User Permissions**: We set up a detailed permissions structure, allowing different levels of access based on the user's role within the company. This ensured that sensitive data was protected while still providing broad access to less critical information.
- **Custom Dashboards**: We created custom dashboards tailored to different departments, making it easy for team members to find and interpret the data most relevant to their roles.
- **Training and Support**: We provided comprehensive training sessions to ensure that all team members could effectively use Metabase, regardless of their technical background.

# The Outcome
The implementation of Metabase transformed the way our client interacted with their data. Team members who previously had no access to data could now easily explore and analyze information relevant to their work. This democratization of data access led to more informed decision-making across the organization and fostered a culture of curiosity and data-driven thinking.

# Conclusion
By leveraging Metabase, Empathy Works helped our client unlock the full potential of their data, making it accessible and actionable for everyone on their team. This case study highlights our commitment to providing innovative solutions that empower our clients and drive meaningful results.



Unlocking Data Access with Metabase: A Case Study

# Introduction
At Empathy Works, we love engaging in projects that make a difference, especially when they're as innovative and inspiring as those led by the Melt Collective. This student organization at the University of British Columbia is passionate about building a circular economy on campus. They focus on reusing different types of waste materials and wanted a way to showcase their experiments and encourage others to get involved.

# The Challenge
The Melt Collective faced a few challenges:

1. **Showcasing Experiments**: They needed a platform to display their various experiments with reusing waste materials.
1. **Tool Accessibility**: They wanted to create a tool library to inventory and keep track of publicly available tools, making them accessible to students for experimentation.
1. **Building Awareness**: Raising awareness about the benefits of a circular economy within the university community was crucial.

# The Solution
We were excited to partner with the Melt Collective on this side project. Our goal was to create something that was both functional and engaging:

1. **Interactive Website**: We developed an interactive website that showcased their experiments with waste materials. This platform included detailed descriptions, photos, and videos of each experiment.
2. **Tool Library**: The centerpiece of our project was the creation of a tool library. We organized and inventoried all the publicly available tools on the University of British Columbia campus, providing guides on how to use the tools and contact information for access. This democratized tool access, allowing more students to experiment with circular materials.
3. **Educational Content**: We added educational content about the circular economy, including articles, infographics, and resources to help visitors understand the importance and impact of their efforts.

# Technology Behind the Solution
We leveraged the power of WordPress to build the website, using several powerful plugins to enhance its capabilities. These plugins allowed us to create a robust inventory system for the tool library and showcase all the different projects visually. This setup made it easier for members of the Melt Collective to contribute to the website and showcase their work.

# The Outcome
The project turned out to be a fun and rewarding experience for everyone involved:

- **Increased Engagement**: The interactive website attracted a lot of interest from students and faculty, who were excited to see the experiments and explore the tool library.
- **Community Building**: The Melt Collective saw a significant increase in participation and collaboration, helping to build a strong community around the concept of a circular economy.
- **Raised Awareness**: The educational content helped raise awareness about the benefits of reusing waste materials and encouraged more sustainable practices on campus.

# Conclusion
Working with the Melt Collective was a fantastic opportunity for Empathy Works to support a great cause. This project not only helped showcase innovative ways to build a circular economy but also brought together a community passionate about sustainability. We're proud to have been a part of this journey and look forward to seeing how the Melt Collective continues to inspire change on campus.



 {/* 
 <ProjectCard url="https://melt.empathy.works" title="MELT Collective" subtitle="Helping build a radial trans-disciplinary circular economy" image="/assets/projects/melt-screenshot-3.png" logo="/assets/projects/melt.png" buttonLabel="View Project">
 We partnered with MELT Collective - a student-led recycling workshop and laboratory based out of UBC campus - to build a portal that helps students to upcycle and reusue waste material.
 The portal includes several interactive exhibits that visualize the buildup of plastic waste and the impact our actions can have on reducing it, the role new biomaterials can have on reducing waste, and steps individuals can take to reduce the footprint of their communities. The portal also features the Tool Library, a directory of all of the manufacturing tools located on the UBC campus along with instructions on how to access them.
 </ProjectCard>
 */}

Exploring Circular Economy Solutions: A Case Study with the Melt Collective


# Introduction
At Empathy Works, we pride ourselves on developing innovative solutions tailored to our clients' unique needs. One of our recent projects involved working with a client who provides assessment services aimed at optimizing the environmental friendliness of buildings. This client sought our expertise to improve their existing system for training, certifying experts, and connecting them to building owners.


# The Challenge
Our client had a comprehensive system in place for training and certifying experts who assess building performance and provide recommendations for environmental optimization. However, their process was highly manual, leading to inefficiencies and challenges:

- **Manual Processes**: Handling training, certification, and maintaining a list of active assessors was time-consuming and prone to errors.
- **Training Delivery**: The existing training material was not being delivered as effectively as possible.
- **Expert-Client Connections**: The process of connecting certified experts with building owners needed to be more streamlined and automated.

# The Solution
Empathy Works partnered with the client to revamp their entire system, focusing on enhancing training delivery, automating processes, and improving client-expert connections. Our approach included:

1. **LearnDash LMS Integration**: We selected LearnDash as the Learning Management System (LMS) due to its robust, off-the-shelf capabilities. LearnDash allowed us to quickly build core functionality, including content delivery, user progress tracking, exams, and certification, without extensive custom development.
2. **Seamless Authentication Integration**: We deeply integrated LearnDash into the client’s existing authentication system. This ensured users could securely access training content without needing to create new accounts, simplifying the process and enhancing user experience.
3. **Automated Certification System**: Using LearnDash, we automated the certification process, tracking expiration dates and notifying experts about renewals. This system also facilitated easy verification of credentials by clients.
4. **Streamlined Expert Management**: We built a centralized database to automatically maintain and update the list of active assessors, ensuring all information was current and accurate.
5. **Expert-Client Matching**: We developed a sophisticated matching algorithm that connected building owners with the most suitable certified experts based on specific needs and geographical location.

# The Outcome
The new system dramatically improved the efficiency and effectiveness of the client’s operations:

- **Enhanced Training**: The interactive platform, powered by LearnDash, led to higher engagement and better knowledge retention among experts.
- **Automated Processes**: The shift from manual to automated processes reduced errors, saved time, and ensured that all certifications and expert lists were up-to-date.
- **Efficient Matching**: The expert-client matching system facilitated quicker and more accurate connections, leading to increased satisfaction for both building owners and experts.
- **Cost-Effective and Quick Implementation**: By leveraging off-the-shelf tools like LearnDash, we delivered a cost-effective solution that met all the client's needs promptly.

# Conclusion
By redesigning the training, certification, and matching systems using efficient, off-the-shelf tools, Empathy Works helped our client deliver higher-quality services and foster better connections between experts and building owners. This case study underscores our commitment to creating tailored solutions that drive significant improvements for our clients.

Enhancing Training and Certification Systems for Environmental Assessors


# Introduction
Empathy Works specializes in developing tailored solutions that enhance our clients' operations and effectiveness. One of our recent projects involved working with a client who provides building assessments and conducts various educational events, including webinars, in-person training, and workshop sessions. The client needed a more efficient and streamlined event management system to support their educational initiatives without disrupting their existing use of ON24.

# The Challenge
Our client faced several challenges with their existing event management setup:

1. **Fragmented Event Sharing and Registration**: While ON24 was effective for streaming and broadcasting events, it lacked seamless integration with their marketing website for sharing upcoming events and facilitating registrations.
2. **Multiple Registration Types**: The client wanted to offer both in-person and virtual events, with the ability to mix these registration types as needed.
3. **Targeted Events**: They required the ability to target specific groups of users, such as certified users or particular user segments, for certain events.
4. **Premium Events Integration**: The client wanted to offer premium events and tie these into their e-commerce system for a streamlined payment process.
5. **Continuing Education Credits**: They needed an automated system for distributing continuing education credits to participants who completed the required objectives.

# The Solution
Empathy Works developed a comprehensive solution that seamlessly integrated with ON24’s APIs while adding the needed functionality:

1. **Seamless Event Integration**: We integrated the event management system with ON24’s APIs, enabling the client to share upcoming events on their marketing website easily and facilitate registrations directly from the site.
2. **Flexible Registration Options**: The system supported multiple registration types, allowing for both in-person and virtual events, and provided the flexibility to mix these as required.
3. **Targeted Event Management**: We built features that enabled the client to target events to specific user groups within their system, ensuring the right audience was reached.
4. **Premium Events and E-Commerce**: The integration allowed the client to offer premium events with associated costs, seamlessly tied into their existing e-commerce system for a smooth transaction process.
5. **Automated Continuing Education Credits**: The system automated the distribution of continuing education credits, simplifying the process and reducing manual effort.

# The Outcome
The implementation of the new event management system resulted in significant improvements:

- **Streamlined Operations**: The integration with ON24 and the automated processes saved time and reduced errors, allowing staff to focus on more strategic tasks.
- **Improved User Experience**: The ability to easily share and register for events, along with targeted event management, enhanced user engagement and satisfaction.
- **Enhanced Flexibility**: Offering both in-person and virtual events, as well as premium events, provided more options for users and helped meet diverse needs.
- **Efficient Credit Distribution**: Automating the distribution of continuing education credits ensured participants received their credits promptly and accurately.

# Conclusion
By enhancing their event management system with seamless ON24 integration and additional functionalities, Empathy Works enabled our client to deliver their educational initiatives more effectively and efficiently. This case study underscores our commitment to creating solutions that drive meaningful improvements for our clients.

Enhancing Event Management with ON24 Integration: A Case Study

A few years ago, I had an opportunity that profoundly touched my heart and left a lasting impression on me. Two of my friends run a non-profit organization that collaborates with an orphanage in South Africa called the Botshabelo. This orphanage provides a home and hope to children from the region, many of whom have very few opportunities for the future. The non-profit’s mission is to create an environment that opens up as many options as possible for these children.

My friends reached out to me with an incredible idea: building a computer lab for the orphanage. Given how much my own life has been transformed by having access to computers from an early age, I was thrilled by the prospect. I knew this could be a game-changer for the orphans, offering them new avenues for learning and personal growth.

One of the challenges we faced was the limited internet connectivity at the orphanage. While it was important to have some offline capabilities, our primary focus was on creating a functional and engaging computer lab. We began by sourcing affordable yet reliable computers. We knew we needed machines that could withstand the rigors of daily use and operate efficiently in an environment where technical support might not always be readily available. Through donations and some savvy shopping, we were able to gather a set of computers that met our requirements.

Next, we focused on the software. We selected programs and educational resources that could be installed locally, reducing the dependency on internet access. This included a mix of educational games, learning tools, and software for teaching basic computer skills. We also set up a local server to host additional resources that the children could access without needing an internet connection.

Setting up the lab was a collaborative effort. Volunteers from the local community joined us, and we worked together to transform a modest room into a fully functioning computer lab. The excitement and anticipation from the children were palpable, and their enthusiasm was a constant source of motivation.

One of the most remarkable aspects of this project was seeing how quickly the children adapted to using the computers. For many of them, working with a keyboard, mouse, and navigating an operating system was a brand new experience. Despite the initial challenges, their interest and determination were evident from the very beginning. This experience highlighted the impact of smartphones in introducing technology to these children. While they had limited access to smartphones and some familiarity with touchscreens, using a computer was an entirely new adventure.

It was fascinating to observe the differences between smartphones and computers in terms of usage. Smartphones are often used for consuming content, whereas computers provided the children with tools for creating and producing. Different students gravitated towards various aspects of computer use. Some were particularly interested in Excel and Word, seeing them as valuable skills for future job opportunities. Others were drawn to design and graphics tools, envisioning opportunities in illustration and digital art. A few students were especially excited about video editing tools, recognizing the creative potential they offered. And, of course, everyone loved playing Pinball.

The moment that stands out the most for me was the first time I demonstrated Wikipedia. We found a way to get an offline downloaded copy of Wikipedia on all the computers. I'll never forget opening up Wikipedia for the first time with the children and exploring articles about Nelson Mandela, Southeast Africa, and other topics. As we continued to explore, we opened an article on Lionel Messi. The children's eyes widened with amazement as they read about his achievements and saw pictures of him in action. You could see the realization dawn on them—the understanding that these computers held access to an immense world of information. It was a powerful moment that underscored the transformative potential of the technology we were providing.

The day we unveiled the new computer lab was one of the most rewarding moments of my life. Watching the children’s faces light up as they explored the computers for the first time was incredibly moving. They immediately began experimenting with the software, playing educational games, and practicing their typing skills. It was clear that this lab would open up a world of possibilities for them.

This experience reinforced my belief in the power of technology to change lives. By giving these children access to computers, we provided them with tools to learn, grow, and explore new opportunities. The Botshabelo now had a new resource to support their mission of creating brighter futures for these kids.

Reflecting on this project, I am immensely grateful for the chance to contribute to such a meaningful cause. It was a reminder of how fortunate I have been to have access to technology and how important it is to extend those opportunities to others. The computer lab at the Botshabelo stands as a testament to what can be achieved when people come together with a shared vision and a commitment to making a difference.


 {/*
 <ProjectCard url="/work/learn-to-program" title="Teaching programming to orphans in South Africa" project="Computer Lab" years="2019" image="/assets/work/computer-lab.jpg" logo="/assets/projects/learn-to-program.png" invertLayout={true} buttonLabel="Explore the Course">
 Learn to Program" was designed in preparation for a month-long volunteer project at an orphanage in South Africa. During the trip, we built a computer lab and taught students the fundamentals of computers, starting with how to use a keyboard and mouse and ending off with basic programming skills.
 This course starts from the very beginning - how does a computer work? From there, we learn how to give instructions to computers, how to make things move on the screen, and how to interact with your programs. By the end of the course, students have the skills they need to create basic games.
 </ProjectCard> */}

A few years ago, the author was invited by friends running a non-profit to help build a computer lab for the Botshabelo in South Africa. This orphanage aims to provide a hopeful future for children with limited opportunities. The author, inspired by their own transformative experience with early access to computers, eagerly accepted the challenge. They faced issues such as limited internet connectivity, leading them to source durable computers and offline educational software. Community volunteers helped set up the lab, transforming a modest room into a fully functioning educational space. The unveiling was a joyous event, with children enthusiastically engaging with the new technology. The project underscored the power of technology to change lives and highlighted the importance of providing such opportunities to those in need. The author felt immensely grateful for contributing to this impactful cause.

Building a computer lab for an orphanage in rural South Africa

Welcome to the first episode of tools we use! This is a new series that highlights the indispensable apps that we use at Empathy on a daily basis.

![Raycast](/assets/blog/raycast.png)

Raycast is an app launcher. It runs in the background and makes it easy to launch other apps. The Windows Start menu is the most famous application launcher. Apple's Spotlight tool has been built into the Mac and iPhone for over a decade now.

So why use (and pay) for a third party tool? Flexibility and power, and Raycast offers both in spades.

Raycast goes beyond launching apps - it also lets you perform actions in them. Let's say I want to play music on a particular Sonos speaker. I can open the Sonos app and select the speakers I want to play music from. Or, I can tell Raycast to send music to a speaker with one command, with no need to launch the Sonos app.

<video width="1310" height="820" controls poster="/assets/blog/raycast-video.png" autoplay="true">
 <source src="/assets/blog/raycast.mp4" type="video/mp4" />
</video>


This power comes from Raycast's extensions. They let developers write commands that hook into other programs. Most popular applications have extensions, and I get a lot of joy from adding new ones to my toolkit.

Some of these extensions go beyond app shortcuts. The pro version offers a built-in AI chatbot that you can access with one keystroke. I’m a paid ChatGPT user, but the convenience of one-click access to a chatbot makes this my got-to for short queries. The paid version also has a built-in translation tool, clipboard history, and can sync your extensions and customizations.

Programmers constantly search for ways to speed up tasks and reduce mouse use. Raycast is designed from the ground up to do this, and is a must-have in my toolkit.

Raycast is a powerful app launcher that speeds up everyday tasks. Its integration with popular apps and built-in AI makes it a must-have tool for developers at Empathy.

Tools we use: Raycast

![A house being split in two](/assets/blog/splitting-django-in-two.webp)

We recently simplified an application that had grown a little too big for its own good. What started off as a single text field on a data model had grown into a separate model, then eventually a series of models and views. We eventually found ourselves working on a monolithic application, with the majority of our code living in a couple of massive files.

Refactoring this on an app that has not yet been released is pretty straightforward, but it gets much more tricky when the site is live and has significant traffic. We wanted a solution that was automated and easy to test so we could be totally sure that the migration would work flawlessly on our production site.

We were already familiar with how Django's migation system makes it easy to changes to data models, but it turns out it's also great at more complex data migrations as well. We wanted to automate the data migration process so that everything would be migrated in one command; Django migrations are executed sequentially, can be automatically run on deployment, can be rolled back if there are failures, and know when each step is completed, which makes them ideal for more complex refactoring.

# Splitting the code up
We started by creating a new Django app and moving the relevant `models.py`, `views.py`, `urls.py`, `admin.py`, `forms.py` and other related code in the new app. We then updated all references to point to the new locations. Many IDEs have built-in refactoring tools that handle the process of renaming import statements for us. Pretty straightforward so far!
 
If you are working with an app that is under development, you are pretty much ready to go at this point. Simply run `manage.py makemigrations` to create new model migrations, and you should be ready to go. 

Unfortunately, it's a little more complicated with a live app.

# Moving live data models to a new app manually
If you are comfortable with manually migrating the data, you can take advantage of Django's built-in `dumpdata` and `loaddata` management commands to copy over data from the old modesl to the new. This works well if you are comfortable taking the website offline for a while, running `dumpdata` before the migrations are run, and then loading the new data post-migration. 

We decided against this approach because of the un-automated nature of this process. Instead, we wanted something we could run multiple times on our local and staging servers until we were confident that the migration would work perfectly.

# Moving live data with data migrations
We handled the migration over a series of data migrations steps. Here are the steps we took:

**1. Create a copy of each model you are migrating in the new app**

Contrary to what might seem logical, you shouldn't delete models from the old app immediately. Instead, create a copy of those models into your new app and run the initial data migration. You should now have two identical models - one in the old app and one in the new.

**2. Copy data into the new models via data migrations**

Next, create an empty migration file using the command `makemigrations --empty <new_app_name>` management command; this will hold the code that transfers data from the old models to the new ones. This data migration should iterate through the instances of the old data model and create them in the new model:

```python
# 000x_data_migration.py

def copy_data(apps, schema_editor):
 OldModel = apps.get_model('old_app', 'OldModel')
 NewModel = apps.get_model('new_app', 'NewModel')
 for old_model_instance in OldModel.objects.all():
 new_model_instance = NewModel(
 field1=old_model_instance.field1,
 field2=old_model_instance.field2,
 ...
 )
 new_model_instance.save()

class Migration(migrations.Migration):
 # ...
 operations = [
 migrations.RunPython(copy_data),
 ]
```
 
**3. Migrate references to the new models**

Your application may have models that reference the models you are moving to the new app. If so, create an additional field on those models that reference the new models:
 
```python
class DependantModel(models.Model):
 foreign_key_field = models.ForeignKey(OldModel)
 
 # new field to copy data from old model
 new_foreign_key_field = models.ForeignKey(NewModel)
```

Next, run run `makemigrations <dependant_app_name>` to create the migration file for the new field. Within this migration file, add a data migration function that transfers data from the old model to the new one:

```python
# dependant app migrations
from django.db import migrations, models

def copy_data(apps, schema_editor):
 DependantModel = apps.get_model('dependant_app', 'DependantModel')
 DependantModel.objects.update(new_foreign_key_field=F('foreign_key_field'))

class Migration(migrations.Migration):
 # ...
 operations = [
 migrations.RunPython(copy_data),
 ]
 ```

After data is migrated, remove the old foreign key field and rename the new field to match the old field's name. This ensures continuity for dependent apps using the same field name:

```python
class Migration(migrations.Migration):
 operations = [
 migrations.RunPython(copy_data),
 migrations.RemoveField(
 model_name='dependant_model',
 name='foreign_key_field',
 ),
 migrations.RenameField(
 model_name='dependant_model',
 old_name='new_foreign_key_field',
 new_name='foreign_key_field',
 ),
 ]
```
 
**4. Create migrations to remove the old models**

Remove old models from the old app, then create a migration file to delete the old models using the command `makemigrations <old_app_name>`.

**5. Apply migrations**

Apply the migrations using the command `python manage.py migrate`. 

Voila, you have automated the process of creating the new data models, copying over the data to them, and deleting the old models. The `migrate` command should handle all of this for you, in a way that is easy to test and repeat on production.

# Testing your migrations
The big advantage of this approach is that it makes it much easier to validate that your migrations are working correctly, as there is no human involvement in the migration process and the behaviour should work the exact same on staging and production. In addition to manually testing the application post-migration, there are a couple additional steps you can take to ensure that the data was copied correctly:

 - Use assertions in the data migration to ensure that the data was copied correctly. This allows you to break the migration flow prior to deletion commands being run if data has not been propertly copied over correctly
 - Check the data in the new models to ensure that it matches the data in the old models.
 - Check the dependant apps to ensure that the data was copied correctly and that the application is working as expected.
 
It's always a good idea to have a backup of the database before running any data migration so that you can restore the it if anything goes wrong. You should also run the migration commands on a staging server that has a recent copy of your production database so you are not caught out by irregularities with real data.
 
# Complex data fields

### DateTimeFields

If you have any kind of auto generated time stamps in your old models (like `created_at` with `auto_now_add=True` and `updated_at` with `auto_now=True`), and you want to keep them as is without getting them updated, you need to make those fields nullable by adding `null=True` and `blank=True` to the fields in the new models.
 
Once you have copied the data from the old models to the new models, you can remove the `null=True` and `blank=True` from the fields in the new models and add the `auto_now_add=True` and `auto_now=True` to the fields in the new models.

### ManyToManyFields

ManyToManyFields are a bit tricky to handle. ManyToManyFields are represented by an intermediate table that has a foreign key to both the models that are related. When you copy the data from the old models to the new models, this data is not copied by default; when you remove the old models, the data in the intermediate table will be removed as well. 

To handle this, you can get access to the through model of the newly created ManyToManyField and copy the data from the old models to the new models in the through model as well.

```python
# 0002_copy_data.py
ThroughModel = NewModel.m2m_field.through
for old_model_instance in OldModel.objects.all():
 new_model_instance = NewModel(
 field1=old_model_instance.field1,
 field2=old_model_instance.field2,
 ...
 )
 new_model_instance.save()
 for m2m_instance in old_model_instance.m2m_field.all():
 ThroughModel.objects.create(
 new_model=new_model_instance,
 m2m_field=m2m_instance
 )
```

### FileFields

If you have any FileFields in your old models, you need to copy the files from the old models to the new models as well. One way to do this is to use the Django `FileField`'s `save` method to copy the files from the old models to the new models. Note that this will use the file path of the old models to name the files in the new models. If you want to change this, use `os.path.basename` to get the file name and use it to save the file in the new models.

```python
# 0002_copy_data.py
for old_model_instance in OldModel.objects.all():
 new_model_instance = NewModel(
 field1=old_model_instance.field1,
 field2=old_model_instance.field2,
 ...
 )
 new_model_instance.save()
 if old_model_instance.file_field:
 new_model_instance.file_field.save(
 os.path.basename(old_model_instance.file_field.name),
 old_model_instance.file_field
 )
```

### Auto incrementing IDs

One thing to keep in mind is that the ID sequence of the new models will be different from the old models. To solve this, you can use the `setval` function in PostgreSQL to set the sequence of the new models to the maximum ID of the old models.

```python
# 0002_copy_data.py
...
migrations.RunSQL('SELECT setval(\'new_app_newmodel_id_seq\', (SELECT MAX(id) FROM old_app_oldmodel));'),
```

# Happy migrating!

It can be a bit of a headache to build a migration plan for a live site, but watching it roll out is worth the hassle. We were able to test the migration dozens of times with production data on our staging servers and verify that everything was working properly before the live migration. 

When it came time to run the migration on production, we put the site into maintenance mode for a few minutes, merged the migration into the main branch, and watched a the models and data were migrated and copied over in sequence. 

In the end, Django's migrations system allowed us to make a stressful and complicated process as routine as any other code deployment.

Authors: Sepehr Sobhani and Jean-Marc Skopek

Diving into the complexities of managing a live Django application, we recently embarked on a journey to streamline an overly cumbersome app. Initially, a simple text field evolved into a web of models and views, leading us to tackle the monolithic beast it had become. The challenge was not just refactoring but doing so on a live site bustling with traffic. Our goal? To automate and flawlessly execute the migration without disrupting our users. Through a series of calculated steps, from duplicating models to meticulously migrating data and references, we leveraged Django's migration system's power for a smooth transition. This methodical approach not only ensured the integrity of our live data but also made the entire process repeatable and testable, assuring us of its success on our production site. The adventure into refactoring and migration was a testament to Django's robustness and our commitment to maintaining a seamless user experience, even behind the scenes.

Splitting a live Django application in two


A **M**inimum **V**iable **P**roduct is the simplest, most cut-back version of your idea - what you get when you strip everything out until you have the smallest and simplest product that will have some value. In many cases, it's also how your idea should be introduced to the world.

It's pretty common for an MVP to feel a little bit uncomfortable. You will be tempted to keep working on it, to polish it up, to add more features. You might even feel a little embarrassed about it. That’s completely normal.

The big downside of the MVP is that it will result in a far simpler tool that will feel incomplete (because it is). It will have far fewer features than you wish and will feel like a shadow of your vision. But if you have an idea of what you want to build - the end goal - you can design your MVP so that it acts as a foundation, a platform on which to build all of your additional features.

Your ability to launch an MVPs is really only possible because of the measurement opportunities that you get with software. In most industries, it can be a little tricky to understand how your product or service is being used. 

![International Space Station](/assets/mvp/space-station.gif)
*The first module of the International Space Station - the FGB - was launched in 1998. Since then, additional modules have been launched and updated, with the most recent Prichal docking module being assembled in 2021. Source: [Nasa.gov](https://www.nasa.gov/international-space-station/space-station-facts-and-figures/)*

Let's say you are doing competitive analysis at a grocery store; every experiment requires working with grocery chains to measure purchases, aggregate the information across stores, and make sense of the data. New experiments are costly and time-consuming tasks, taking weeks or even months to run.

MVPs make it easier to make adjustments when things don't go to plan. It's very rare - heck, borderline impossible - for an idea to be perfect the first time around. Your users will react to your tool differently, often forming clusters with different behaviours, and it’s very hard to predict what the behaviour of each cluster will be. In most cases, the most efficient way of optimizing a tool is by launching it to the public, letting users try it, and collecting and iterating on usage measurements.


*I can actually draw some kind of robot body that is the product - a voltron on its own and part of a megazoid. Or a space station - modules; yeah, let’s do that.*

Big software projects take time to build; their complexity results in bugs and a slower development pace; it's harder to remove a feature than it is to add one%    


Launch as early as possible, and build up from there