❗Urgent - Deleted fork environment, all database volumes removed from production 🙃
![alert](https://cdn.discordapp.com/emojis/954734348379238462.gif)
36 Replies
Project ID:
80d54f52-7f6a-415c-b857-cb6aef417c81
80d54f52-7f6a-415c-b857-cb6aef417c81
Thread has been flagged to Railway team by @Brody.
<a:party_railway:812486353853218816> ![pepelove](https://cdn.discordapp.com/emojis/954599893123629116.png)
![hearts](https://cdn.discordapp.com/emojis/804223081014493234.gif)
![pepelove](https://cdn.discordapp.com/emojis/954599893123629116.png)
![hearts](https://cdn.discordapp.com/emojis/804223081014493234.gif)
its still Sunday night for a good chunk of the team, please be patient
No worries, we have created a new database, and restored the backup, we won't be touching the old ones for now.
Happy to be patient.
in the mean time can you provide more clarification? the volumes where deleted off of the service? leaving a database service without a volume?
Correct!
mind sending some screenshots
- All our deploys disappeared, and turned off
- Database volumes deleted
Steps to cause the issue:
- Fork development environment (not production)
- We reconfigured some services (so we wouldn't get duplicates on 2 of our listening services)
- Then decided we didn't need the fork, removed it
- Our monitoring system detected everything was offline
- Checked Railway, all deployments gone on everything (even the items we didn't touch in the fork)
Steps taken since
- Created a new DB, restored backup from 2 days ago (we're still missing 2 days worth of time series data)
- Pointed APIs at the new DB
![No description](https://cdn.answeroverflow.com/1196372648771461140/image.png)
that definitely goes against what the docs says
![No description](https://cdn.answeroverflow.com/1196378459753160774/image.png)
That's what I thought! 🙃
can you show me how you made the fork of the dev env?
Sure
It was 100% a fork
- I saw the arrow icon below the dev environment, pointing to the fork environment in the environments section in app
- When we made a change it said "merge changes" at the bottom