Migrating Data from EEPROM to FLASH devices
by Fridolin Kolb (comments: 0)
Almost all applications in embedded systems have the need to store some data in non volatile memory areas in order to have it available after power loss.
A lot of those systems until now have used external serial EEPROM circuits for that purpose.
Nowadays micro controller suppliers offer their chips with integrated data FLASH areas that are able to replace the additional EEPROM. This is a big advantage in terms of test effort and cost reduction.
But the change from an EEPROM to an internal data FLASH is not as easy as it seems.
In this article I will give an overview of the main differences between an EEPROM and a data FLASH and what an embedded software system has to do to manage data in an internal data FLASH.
I am interested in safety items, so why should I read this?
I know that there are some application notes from different hardware providers about this item, but all of them are very specific for their design and there is no common overview.
Especially the safety aspect is not covered by those publications. I think storing data and data consistency is one of the top functional safety items in embedded systems. You don't think so? OK, so ask the functional safety manager of your trust, how to deal with data loss of safety critical data stored in a non volatile memory, and the item will be on the top five of your most critical items list, believe me.
This is a try to reflect the technical aspects of migration from EEPROM to Data FLASH including also the safety aspects.
EEPROM? Data FLASH? Isn't that the same?
Indeed the technological base of EEPROM and FLASH is the same. Moreover one can say the FLASH technology has been developed out of the EEPROM technology. Both technologies are electrical programmable and erasable read only memories.
But there is one important difference: The term "EEPROM" tends to be used for devices that are erasable on a per byte or per "programming word" basis, whereas the term "FLASH" tends to be used for devices which only support erase capabilities in their entirety or on a sector base (mostly > 1k). This is the main difference, we are talking about. This article uses the terms in "EEPROM" and "FLASH" in this intention.
Unfortunately some chip providers call their byte erasable products "FLASH", just because of marketing reasons, because the term "FLASH" gets more and more popular as synonym for a fast writable memory, what it actually is, as we can see later on.
Coming with that big separating point there are some more technological differences, that influence data management in EEPROM and data FLASH. These are the write times and the erase times. Following table gives an overview:
|Feature / Characteristic||EEPROM||FLASH|
|byte write time||1ms to 10ms||< 100µs|
|erase time (entirely)||5ms to 50ms||from seconds up to minutes|
|erase / programming cycles||from ~100k up to > 1 million||from < 10k up to < 100k|
I think one can see, from what characteristic the term "FLASH" is derived.
Finally, when using an internal data FLASH of a micro controller system, there is one more slightly difference in comparison to an external EEPROM:
An external EEPROM will finish a once started write command as long as the device is powered, even if the micro controller is already in reset state or clock failed.
An internal data FLASH of a micro controller depends on the clock and the reset of the micro and will suspend write or program cycles immediately if one of the signal fails. Furthermore, some microcontrollers will stop parts of the internal peripheral (e.g. timer) during FLASH erase cycles.
This is one point that has to be taken into account e.g. in shutdown and emergency storage situations.
Sounds interesting, but what are the consequences?
For non volatile data management these differences have a huge impact. To see how, let us construct a scenario for non volatile data management and have a look on how this scenario may be handled using an EEPROM or on the other hand using an internal data FLASH.
Let us construct the following use case:
We have 3 data items which shall be stored in a non volatile memory area: NVData0, NVData1 and NVData2. The length shall be 1 byte for NVData0, 2 byte for NVData1 and 4 byte for NVData3.
At least one of our data items shall be updated cyclically in our EEPROM, but we assume that update rate respects maximum write cycles of the device over lifetime.
And because we are interested in functional safety, let's postulate that all of our 3 data items shall have a safety critical impact in case of data loss.
The EEPROM Approach
Let's start with a look on how non volatile data management usually works using an EEPROM device for our scenario.
For a user EEPROM memories nearly behave like normal RAM memory, if we forget about the timing and internal operations. So it is possible to assign a specific memory address to each memory item. As a consequence we are able to set up a memory map for our three data items to be stored in the EEPROM.
OK we found a place for our data items. But we are still not satisfied, because we have to prevent data loss and data corruption.
Therefore we first need the possibility to detect data corruption. Standard answer to that is to add a CRC to the data item. And why should we change a winning team? We take that solution. The length of the CRC to add strongly depends on the data length and the necessary Hamming Distance1. In our case a CRC8 will be sufficient.
Now one could say, from functional safety point of view everything is fine. We detect damaged data, go to safe state and that's it.
But data damage scenario is a very likely one for cyclically written data e.g. in case of sudden power loss, as you might not be able to assure that there is no write action going on at that point of time. And your customer will be very unsatisfied with your very safe but not available system.
This is an example that safety measures do not automatically increase the quality of a system and vice versa! Unfortunately this is a far spread wrong assumption.
What we need is a mechanism to prevent data loss while keeping the availability as good as possible. Simplest solution is to mirror our data items, which means we have to store the data items twice in our EEPROM. This means, when writing data items, we first update data in the original address and in a second step we update data in the mirror address. If there is a problem occurring during write process (e.g. power loss), there will be always a valid data set either at the original address or at the mirror address.
To bring in additional safety we do not write mirror data directly after original data, but e.g. divide up the complete EEPROM address space in two areas for original and mirror data. This helps us to prevent that our mirror gets corrupt together with the original by EEPROM errors affecting neighbored memory cells.
Following picture shows the resulting EEPROM memory map (assumed EEPROM Length is 256 byte):
This is a fix memory structure, which is easy to handle by memory management. The translation from data item to address is normally done in a layer called EEPROM Abstraction (EA). An example for definition of such a layer can be found in AUTOSAR standard AUTOSAR_SWS_EEPROMAbstraction
The Data FLASH Approach
When using a data FLASH for the same scenario two things fundamentally change:
It is not possible to overwrite data in a fixed address without erasing a big area of memory.
The maximum number of programming cycles for a single cell decreases dramatically (approximately by factor 10).
This leads to the common approach for saving data in an internal data flash:
The Flash EEPROM Emulation (FEE)
There are a lot of algorithms for emulating EEPROM in FLASH. The following is just one of them. The key challenges are the same for all of the possible algorithms. To deal with the characteristics of a data FLASH memory following techniques are usually taken.
The complete FLASH memory area is divided in at least two blocks. One of these blocks is the so called active block. This block keeps the current saved data. The other block is the passive block. This block is empty.
Each data item to be stored has a fixed ID and a fixed length. This is a constant information and fixed at compile time and therefore stored in program memory. Also the start addresses of the data FLASH blocks and the length of the FLASH blocks is fixed at compile time and stored in program memory.
Data saved to the active block is written just one data item behind the other. For that it is necessary to save also the information, where the latest data of a special data item has been written. This management information can be seen as a kind of simple data management system.
To keep data and management information separated, data is written from the start of the active block to the end, on the other side management information is written from the end of the active block to the start. Additionally there must be some header information at the very beginning of each block, which marks the block as active or passive.
When the active block is full, which means data and management information meet somewhere in the middle of the active block, the latest data of each stored data item is copied from the active to the passive block. Afterward the passive block becomes the active block and the formerly active block is erased and becomes the passive block.
At start-up the FEE driver first checks for the active block by reading the header information of the blocks. Then it checks for the latest entry in the active block by checking management information beginning at he block end until it finds an empty entry (erased flash cell). This marks the address for the next management info. The latest management information at the same time delivers the address and the ID of the latest data written to the flash. With that information FEE driver can restore the next address for data to be written to the FLASH. Data management is now initialized and data can be read and written from / to the data FLASH again.
Following picture shows a typical memory mapping of an active FLASH block for our scenario:
Safety Aspects of FEE
Detection of corrupted data: As for the EEPROM approach for the FEE also a CRC may be used to detect corrupted data with the same considerations as already mentioned for EEPROM.
Prevention of data loss: It is also possible to store the data items twice in order to prevent data loss as already mentioned for the EEPROM approach. But it is not really necessary! As all new data is just written to a new flash address all "old" data are still available in the memory. This way there is always a valid copy of the data item available in the FLASH. Only the very first data entry should be written twice, as at that point of time there is no copy of the data item available. This can be handled by FEE driver.
Prevention of loss of management information: For management information the same considerations are valid as for data items. This means that management information also should be protected against corruption by a CRC. Additionally the management information can be stored twice (mirrored) to prevent loss of information. I would not suggest to use the available copy of the management information of a older data copy of the same data item. Because in this case you might use the older copy despite of the availability of a correct data item just because of the corruption of the management info.
Conclusion and Consequences
Although EEPROM and FLASH memory are very similar memory technologies, storing data to them requires quite different data management approaches. Especially the change from a fixed memory mapping in the EEPROM to a dynamic mapping with a simple data management in a FLASH makes huge differences. Following consequences should therefore be considered when migrating from an EEPROM to a data FLASH.
The size of the data memory must be adapted: Storing data in a FLASH memory requires much more space than in an EEPROM. This is first due to the fact, that the management information has to be stored also in FLASH. Second aspect influencing memory size is the low number of programming cycles of the FLASH memory. Depending on the size of data to be stored and the store frequency FLASH memory should be 4 to 10 times bigger than an EEPROM.
Split up big data items: As a direct consequence of point 1. big data items stored in EEPROM should be split up into smaller parts, when migrating to FLASH. Big, frequently written data items would lead to a more frequent change between active and passive FLASH blocks because the blocks would be filled faster. Storing smaller blocks could prevent this.
Respect the run time for a block change: The driver must respect the run time for a block change from active to the passive FLASH block. Depending on the number of data items to be copied from one FLASH block to another the driver will be blocked for a longer time for accepting storage commands from an application. A FEE driver should consider this by e.g. bigger command buffer sizes in comparison to an EEPROM driver.
No data mirrors necessary: As there is always a valid copy available for the stored data items there is no necessity of mirrored data. This eases handling of data corruption and prevention of data loss, when using FLASH.
As we can see the migration from an EEPROM to a data FLASH requires a lot of work and care, and doing this fast and easy is indeed a fairy tale.Hopefully I have been able to make clear the main differences between the two memory technologies and the resulting consequences for the application.
If I forgot some aspects of this issue, please feel free to tell about your experiences and solutions. I am always looking forward to learn something new.
The Hamming distance is named after Richard Hamming. A code with minimum Hamming distance d between its codewords can detect at most d-1 errors. ↩︎