wirtz
/
esp-link
mirror of https://github.com/jeelabs/esp-link.git
1
0
Fork 0
Code Issues Releases Wiki Activity

add spiffs code, not yet actually integrated

Browse Source
platformio
Thorsten von Eicken 6 years ago
parent 0a73be2983
commit fef4c67958
No known key found for this signature in database
GPG Key ID: C7F972A59D834B46
23 changed files with 14451 additions and 0 deletions
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Show Stats Download Patch File Download Diff File
  1. 1
      platformio.ini
  2. 20
      src/spiffs/LICENSE
  3. 223
      src/spiffs/README.md
  4. 373
      src/spiffs/default/spiffs_config.h
  5. 816
      src/spiffs/spiffs.h
  6. 319
      src/spiffs/spiffs_cache.c
  7. 1008
      src/spiffs/spiffs_check.c
  8. 369
      src/spiffs/spiffs_config.h
  9. 606
      src/spiffs/spiffs_gc.c
  10. 1451
      src/spiffs/spiffs_hydrogen.c
  11. 2364
      src/spiffs/spiffs_nucleus.c
  12. 842
      src/spiffs/spiffs_nucleus.h
  13. 12
      src/spiffs/test/main.c
  14. 84
      src/spiffs/test/params_test.h
  15. 1266
      src/spiffs/test/test_bugreports.c
  16. 427
      src/spiffs/test/test_check.c
  17. 122
      src/spiffs/test/test_dev.c
  18. 2507
      src/spiffs/test/test_hydrogen.c
  19. 1114
      src/spiffs/test/test_spiffs.c
  20. 109
      src/spiffs/test/test_spiffs.h
  21. 238
      src/spiffs/test/testrunner.c
  22. 165
      src/spiffs/test/testrunner.h
  23. 15
      src/spiffs/test/testsuites.c

1
platformio.ini
Unescape View File

@ -5,6 +5,7 @@ env_default = esp12e
platform = espressif8266 platform = espressif8266
board = esp12e board = esp12e
framework = esp8266-nonos-sdk framework = esp8266-nonos-sdk
src_filter = +<*> -<*/test/> -<*/*/test/>
monitor_speed = 115200 monitor_speed = 115200
upload_resetmethod = ck upload_resetmethod = ck
;build_flags = -Wl,-Teagle.flash.1m0.ld ;build_flags = -Wl,-Teagle.flash.1m0.ld

20
src/spiffs/LICENSE
Unescape View File

@ -0,0 +1,20 @@
The MIT License (MIT)
Copyright (c) 2013-2017 Peter Andersson (pelleplutt1976<at>gmail.com)
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

223
src/spiffs/README.md
Unescape View File

@ -0,0 +1,223 @@
# SPIFFS (SPI Flash File System)
**V0.3.7**
## ESP-LINK
The spiffs code has been copied from [Peter Andersson's
repository](https://github.com/pellepl/spiffs) and adapted to the use in esp-link.
The only file modified is `spiffs_config.h`.
## Original README
[![Build Status](https://travis-ci.org/pellepl/spiffs.svg?branch=master)](https://travis-ci.org/pellepl/spiffs)
Copyright (c) 2013-2017 Peter Andersson (pelleplutt1976 at gmail.com)
For legal stuff, see [LICENSE](https://github.com/pellepl/spiffs/blob/master/LICENSE). Basically, you may do whatever you want with the source. Use, modify, sell, print it out, roll it and smoke it - as long as I won't be held responsible.
Love to hear feedback though!
## INTRODUCTION
Spiffs is a file system intended for SPI NOR flash devices on embedded targets.
Spiffs is designed with following characteristics in mind:
- Small (embedded) targets, sparse RAM without heap
- Only big areas of data (blocks) can be erased
- An erase will reset all bits in block to ones
- Writing pulls one to zeroes
- Zeroes can only be pulled to ones by erase
- Wear leveling
## BUILDING
`mkdir build; make`
Otherwise, configure the `builddir` variable towards the top of `makefile` as something opposed to the default `build`. Sanity check on the host via `make test` and refer to `.travis.yml` for the official in-depth testing procedure. See the wiki for [integrating](https://github.com/pellepl/spiffs/wiki/Integrate-spiffs) spiffs into projects and [spiffsimg](https://github.com/nodemcu/nodemcu-firmware/tree/master/tools/spiffsimg) from [nodemcu](https://github.com/nodemcu) is a good example on the subject.
## FEATURES
What spiffs does:
- Specifically designed for low ram usage
- Uses statically sized ram buffers, independent of number of files
- Posix-like api: open, close, read, write, seek, stat, etc
- It can run on any NOR flash, not only SPI flash - theoretically also on embedded flash of a microprocessor
- Multiple spiffs configurations can run on same target - and even on same SPI flash device
- Implements static wear leveling
- Built in file system consistency checks
- Highly configurable
What spiffs does not:
- Presently, spiffs does not support directories. It produces a flat structure. Creating a file with path *tmp/myfile.txt* will create a file called *tmp/myfile.txt* instead of a *myfile.txt* under directory *tmp*.
- It is not a realtime stack. One write operation might last much longer than another.
- Poor scalability. Spiffs is intended for small memory devices - the normal sizes for SPI flashes. Going beyond ~128Mbyte is probably a bad idea. This is a side effect of the design goal to use as little ram as possible.
- Presently, it does not detect or handle bad blocks.
- One configuration, one binary. There's no generic spiffs binary that handles all types of configurations.
## NOTICE
0.4.0 is under construction. This is a full rewrite and will change the underlying structure. Hence, it will not be compatible with earlier versions of the filesystem. The API is the same, with minor modifications. Some config flags will be removed (as they are mandatory in 0.4.0) and some features might fall away until 0.4.1. If you have any worries or questions, it can be discussed in issue [#179](https://github.com/pellepl/spiffs/issues/179)
## MORE INFO
See the [wiki](https://github.com/pellepl/spiffs/wiki) for [configuring](https://github.com/pellepl/spiffs/wiki/Configure-spiffs), [integrating](https://github.com/pellepl/spiffs/wiki/Integrate-spiffs), [using](https://github.com/pellepl/spiffs/wiki/Using-spiffs), and [optimizing](https://github.com/pellepl/spiffs/wiki/Performance-and-Optimizing) spiffs.
For design, see [docs/TECH_SPEC](https://github.com/pellepl/spiffs/blob/master/docs/TECH_SPEC).
For a generic spi flash driver, see [this](https://github.com/pellepl/spiflash_driver).
## HISTORY
### 0.3.7
- fixed prevent seeking to negative offsets #158
- fixed file descriptor offsets not updated for multiple fds on same file #157
- fixed cache page not closed for removed files #156
- fixed a lseek bug when seeking exactly to end of a fully indexed first level LUT #148
- fixed wear leveling issue #145
- fixed attempt to write out of bounds in flash #130,
- set file offset when seeking over end #121 (thanks @sensslen)
- fixed seeking in virgin files #120 (thanks @sensslen)
- Optional file metadata #128 (thanks @cesanta)
- AFL testing framework #100 #143 (thanks @pjsg)
- Testframe updates
New API functions:
- `SPIFFS_update_meta, SPIFFS_fupdate_meta` - updates metadata for a file
New config defines:
- `SPIFFS_OBJ_META_LEN` - enable possibility to add extra metadata to files
### 0.3.6
- Fix range bug in index memory mapping #98
- Add index memory mapping #97
- Optimize SPIFFS_read for large files #96
- Add temporal cache for opening files #95
- More robust gc #93 (thanks @dismirlian)
- Fixed a double write of same data in certain cache situations
- Fixed an open bug in READ_ONLY builds
- File not visible in SPIFFS_readdir #90 (thanks @benpicco-tmp)
- Cache load code cleanup #92 (thanks @niclash)
- Fixed lock/unlock asymmetry #88 #87 (thanks @JackJefferson, @dpruessner)
- Testframe updates
New API functions:
- `SPIFFS_ix_map` - map index meta data to memory for a file
- `SPIFFS_ix_unmap` - unmaps index meta data for a file
- `SPIFFS_ix_remap` - changes file offset for index metadata map
- `SPIFFS_bytes_to_ix_map_entries` - utility, get length of needed vector for given amount of bytes
- `SPIFFS_ix_map_entries_to_bytes` - utility, get number of bytes a vector can represent given length
New config defines:
- `SPIFFS_IX_MAP` - enable possibility to map index meta data to memory for reading faster
- `SPIFFS_TEMPORAL_FD_CACHE` - enable temporal cache for opening files faster
- `SPIFFS_TEMPORAL_CACHE_HIT_SCORE` - for tuning the temporal cache
### 0.3.5
- Fixed a bug in fs check
- API returns actual error codes #84) (thanks @Nails)
- Fix compiler warnings for non-gcc #83 #81 (thanks @Nails)
- Unable to recover from full fs #82 (thanks @rojer)
- Define SPIFFS_O_* flags #80
- Problem with long filenames #79 (thanks @psjg)
- Duplicate file name bug fix #74 (thanks @igrr)
- SPIFFS_eof and SPIFFS_tell return wrong value #72 (thanks @ArtemPisarenko)
- Bunch of testframe updates #77 #78 #86 (thanks @dpreussner, @psjg a.o)
### 0.3.4
- Added user callback file func.
- Fixed a stat bug with obj id.
- SPIFFS_probe_fs added
- Add possibility to compile a read-only version of spiffs
- Make magic dependent on fs length, if needed (see #59 & #66) (thanks @hreintke)
- Exposed SPIFFS_open_by_page_function
- Zero-size file cannot be seek #57 (thanks @lishen2)
- Add tell and eof functions #54 (thanks @raburton)
- Make api string params const #53 (thanks @raburton)
- Preserve user_data during mount() #51 (thanks @rojer)
New API functions:
- `SPIFFS_set_file_callback_func` - register a callback informing about file events
- `SPIFFS_probe_fs` - probe a spi flash trying to figure out size of fs
- `SPIFFS_open_by_page` - open a file by page index
- `SPIFFS_eof` - checks if end of file is reached
- `SPIFFS_tell` - returns current file offset
New config defines:
- `SPIFFS_READ_ONLY`
- `SPIFFS_USE_MAGIC_LENGTH`
### 0.3.3
**Might not be compatible with 0.3.2 structures. See issue #40**
- Possibility to add integer offset to file handles
- Truncate function presumes too few free pages #49
- Bug in truncate function #48 (thanks @PawelDefee)
- Update spiffs_gc.c - remove unnecessary parameter (thanks @PawelDefee)
- Update INTEGRATION docs (thanks @PawelDefee)
- Fix pointer truncation in 64-bit platforms (thanks @igrr)
- Zero-sized files cannot be read #44 (thanks @rojer)
- (More) correct calculation of max_id in obj_lu_find #42 #41 (thanks @lishen2)
- Check correct error code in obj_lu_find_free #41 (thanks @lishen2)
- Moar comments for SPIFFS_lseek (thanks @igrr)
- Fixed padding in spiffs_page_object_ix #40 (thanks @jmattsson @lishen2)
- Fixed gc_quick test (thanks @jmattsson)
- Add SPIFFS_EXCL flag #36
- SPIFFS_close may fail silently if cache is enabled #37
- User data in callbacks #34
- Ignoring SINGLETON build in cache setup (thanks Luca)
- Compilation error fixed #32 (thanks @chotasanjiv)
- Align cand_scores (thanks @hefloryd)
- Fix build warnings when SPIFFS_CACHE is 0 (thanks @ajaybhargav)
New config defines:
- `SPIFFS_FILEHDL_OFFSET`
### 0.3.2
- Limit cache size if too much cache is given (thanks pgeiem)
- New feature - Controlled erase. #23
- SPIFFS_rename leaks file descriptors #28 (thanks benpicco)
- moved dbg print defines in test framework to params_test.h
- lseek should return the resulting offset (thanks hefloryd)
- fixed type on dbg ifdefs
- silence warning about signed/unsigned comparison when spiffs_obj_id is 32 bit (thanks benpicco)
- Possible error in test_spiffs.c #21 (thanks yihcdaso-yeskela)
- Cache might writethrough too often #16
- even moar testrunner updates
- Test framework update and some added tests
- Some thoughts for next gen
- Test sigsevs when having too many sectors #13 (thanks alonewolfx2)
- GC might be suboptimal #11
- Fix eternal readdir when objheader at last block, last entry
New API functions:
- `SPIFFS_gc_quick` - call a nonintrusive gc
- `SPIFFS_gc` - call a full-scale intrusive gc
### 0.3.1
- Removed two return warnings, was too triggerhappy on release
### 0.3.0
- Added existing namecheck when creating files
- Lots of static analysis bugs #6
- Added rename func
- Fix SPIFFS_read length when reading beyond file size
- Added reading beyond file length testcase
- Made build a bit more configurable
- Changed name in spiffs from "errno" to "err_code" due to conflicts compiling in mingw
- Improved GC checks, fixed an append bug, more robust truncate for very special case
- GC checks preempts GC, truncate even less picky
- Struct alignment needed for some targets, define in spiffs config #10
- Spiffs filesystem magic, definable in config
New config defines:
- `SPIFFS_USE_MAGIC` - enable or disable magic check upon mount
- `SPIFFS_ALIGNED_OBJECT_INDEX_TABLES` - alignment for certain targets
New API functions:
- `SPIFFS_rename` - rename files
- `SPIFFS_clearerr` - clears last errno
- `SPIFFS_info` - returns info on used and total bytes in fs
- `SPIFFS_format` - formats the filesystem
- `SPIFFS_mounted` - checks if filesystem is mounted

373
src/spiffs/default/spiffs_config.h
Unescape View File

@ -0,0 +1,373 @@
/*
* spiffs_config.h
*
* Created on: Jul 3, 2013
* Author: petera
*/
#ifndef SPIFFS_CONFIG_H_
#define SPIFFS_CONFIG_H_
// ----------- 8< ------------
// Following includes are for the linux test build of spiffs
// These may/should/must be removed/altered/replaced in your target
#include "params_test.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stddef.h>
#include <unistd.h>
#ifdef _SPIFFS_TEST
#include "testrunner.h"
#endif
// ----------- >8 ------------
// compile time switches
// Set generic spiffs debug output call.
#ifndef SPIFFS_DBG
#define SPIFFS_DBG(_f, ...) //printf(_f, ## __VA_ARGS__)
#endif
// Set spiffs debug output call for garbage collecting.
#ifndef SPIFFS_GC_DBG
#define SPIFFS_GC_DBG(_f, ...) //printf(_f, ## __VA_ARGS__)
#endif
// Set spiffs debug output call for caching.
#ifndef SPIFFS_CACHE_DBG
#define SPIFFS_CACHE_DBG(_f, ...) //printf(_f, ## __VA_ARGS__)
#endif
// Set spiffs debug output call for system consistency checks.
#ifndef SPIFFS_CHECK_DBG
#define SPIFFS_CHECK_DBG(_f, ...) //printf(_f, ## __VA_ARGS__)
#endif
// Set spiffs debug output call for all api invocations.
#ifndef SPIFFS_API_DBG
#define SPIFFS_API_DBG(_f, ...) //printf(_f, ## __VA_ARGS__)
#endif
// Defines spiffs debug print formatters
// some general signed number
#ifndef _SPIPRIi
#define _SPIPRIi "%d"
#endif
// address
#ifndef _SPIPRIad
#define _SPIPRIad "%08x"
#endif
// block
#ifndef _SPIPRIbl
#define _SPIPRIbl "%04x"
#endif
// page
#ifndef _SPIPRIpg
#define _SPIPRIpg "%04x"
#endif
// span index
#ifndef _SPIPRIsp
#define _SPIPRIsp "%04x"
#endif
// file descriptor
#ifndef _SPIPRIfd
#define _SPIPRIfd "%d"
#endif
// file object id
#ifndef _SPIPRIid
#define _SPIPRIid "%04x"
#endif
// file flags
#ifndef _SPIPRIfl
#define _SPIPRIfl "%02x"
#endif
// Enable/disable API functions to determine exact number of bytes
// for filedescriptor and cache buffers. Once decided for a configuration,
// this can be disabled to reduce flash.
#ifndef SPIFFS_BUFFER_HELP
#define SPIFFS_BUFFER_HELP 0
#endif
// Enables/disable memory read caching of nucleus file system operations.
// If enabled, memory area must be provided for cache in SPIFFS_mount.
#ifndef SPIFFS_CACHE
#define SPIFFS_CACHE 1
#endif
#if SPIFFS_CACHE
// Enables memory write caching for file descriptors in hydrogen
#ifndef SPIFFS_CACHE_WR
#define SPIFFS_CACHE_WR 1
#endif
// Enable/disable statistics on caching. Debug/test purpose only.
#ifndef SPIFFS_CACHE_STATS
#define SPIFFS_CACHE_STATS 1
#endif
#endif
// Always check header of each accessed page to ensure consistent state.
// If enabled it will increase number of reads, will increase flash.
#ifndef SPIFFS_PAGE_CHECK
#define SPIFFS_PAGE_CHECK 1
#endif
// Define maximum number of gc runs to perform to reach desired free pages.
#ifndef SPIFFS_GC_MAX_RUNS
#define SPIFFS_GC_MAX_RUNS 5
#endif
// Enable/disable statistics on gc. Debug/test purpose only.
#ifndef SPIFFS_GC_STATS
#define SPIFFS_GC_STATS 1
#endif
// Garbage collecting examines all pages in a block which and sums up
// to a block score. Deleted pages normally gives positive score and
// used pages normally gives a negative score (as these must be moved).
// To have a fair wear-leveling, the erase age is also included in score,
// whose factor normally is the most positive.
// The larger the score, the more likely it is that the block will
// picked for garbage collection.
// Garbage collecting heuristics - weight used for deleted pages.
#ifndef SPIFFS_GC_HEUR_W_DELET
#define SPIFFS_GC_HEUR_W_DELET (5)
#endif
// Garbage collecting heuristics - weight used for used pages.
#ifndef SPIFFS_GC_HEUR_W_USED
#define SPIFFS_GC_HEUR_W_USED (-1)
#endif
// Garbage collecting heuristics - weight used for time between
// last erased and erase of this block.
#ifndef SPIFFS_GC_HEUR_W_ERASE_AGE
#define SPIFFS_GC_HEUR_W_ERASE_AGE (50)
#endif
// Object name maximum length. Note that this length include the
// zero-termination character, meaning maximum string of characters
// can at most be SPIFFS_OBJ_NAME_LEN - 1.
#ifndef SPIFFS_OBJ_NAME_LEN
#define SPIFFS_OBJ_NAME_LEN (32)
#endif
// Maximum length of the metadata associated with an object.
// Setting to non-zero value enables metadata-related API but also
// changes the on-disk format, so the change is not backward-compatible.
//
// Do note: the meta length must never exceed
// logical_page_size - (SPIFFS_OBJ_NAME_LEN + 64)
//
// This is derived from following:
// logical_page_size - (SPIFFS_OBJ_NAME_LEN + sizeof(spiffs_page_header) +
// spiffs_object_ix_header fields + at least some LUT entries)
#ifndef SPIFFS_OBJ_META_LEN
#define SPIFFS_OBJ_META_LEN (0)
#endif
// Size of buffer allocated on stack used when copying data.
// Lower value generates more read/writes. No meaning having it bigger
// than logical page size.
#ifndef SPIFFS_COPY_BUFFER_STACK
#define SPIFFS_COPY_BUFFER_STACK (64)
#endif
// Enable this to have an identifiable spiffs filesystem. This will look for
// a magic in all sectors to determine if this is a valid spiffs system or
// not on mount point. If not, SPIFFS_format must be called prior to mounting
// again.
#ifndef SPIFFS_USE_MAGIC
#define SPIFFS_USE_MAGIC (0)
#endif
#if SPIFFS_USE_MAGIC
// Only valid when SPIFFS_USE_MAGIC is enabled. If SPIFFS_USE_MAGIC_LENGTH is
// enabled, the magic will also be dependent on the length of the filesystem.
// For example, a filesystem configured and formatted for 4 megabytes will not
// be accepted for mounting with a configuration defining the filesystem as 2
// megabytes.
#ifndef SPIFFS_USE_MAGIC_LENGTH
#define SPIFFS_USE_MAGIC_LENGTH (0)
#endif
#endif
// SPIFFS_LOCK and SPIFFS_UNLOCK protects spiffs from reentrancy on api level
// These should be defined on a multithreaded system
// define this to enter a mutex if you're running on a multithreaded system
#ifndef SPIFFS_LOCK
#define SPIFFS_LOCK(fs)
#endif
// define this to exit a mutex if you're running on a multithreaded system
#ifndef SPIFFS_UNLOCK
#define SPIFFS_UNLOCK(fs)
#endif
// Enable if only one spiffs instance with constant configuration will exist
// on the target. This will reduce calculations, flash and memory accesses.
// Parts of configuration must be defined below instead of at time of mount.
#ifndef SPIFFS_SINGLETON
#define SPIFFS_SINGLETON 0
#endif
#if SPIFFS_SINGLETON
// Instead of giving parameters in config struct, singleton build must
// give parameters in defines below.
#ifndef SPIFFS_CFG_PHYS_SZ
#define SPIFFS_CFG_PHYS_SZ(ignore) (1024*1024*2)
#endif
#ifndef SPIFFS_CFG_PHYS_ERASE_SZ
#define SPIFFS_CFG_PHYS_ERASE_SZ(ignore) (65536)
#endif
#ifndef SPIFFS_CFG_PHYS_ADDR
#define SPIFFS_CFG_PHYS_ADDR(ignore) (0)
#endif
#ifndef SPIFFS_CFG_LOG_PAGE_SZ
#define SPIFFS_CFG_LOG_PAGE_SZ(ignore) (256)
#endif
#ifndef SPIFFS_CFG_LOG_BLOCK_SZ
#define SPIFFS_CFG_LOG_BLOCK_SZ(ignore) (65536)
#endif
#endif
// Enable this if your target needs aligned data for index tables
#ifndef SPIFFS_ALIGNED_OBJECT_INDEX_TABLES
#define SPIFFS_ALIGNED_OBJECT_INDEX_TABLES 0
#endif
// Enable this if you want the HAL callbacks to be called with the spiffs struct
#ifndef SPIFFS_HAL_CALLBACK_EXTRA
#define SPIFFS_HAL_CALLBACK_EXTRA 0
#endif
// Enable this if you want to add an integer offset to all file handles
// (spiffs_file). This is useful if running multiple instances of spiffs on
// same target, in order to recognise to what spiffs instance a file handle
// belongs.
// NB: This adds config field fh_ix_offset in the configuration struct when
// mounting, which must be defined.
#ifndef SPIFFS_FILEHDL_OFFSET
#define SPIFFS_FILEHDL_OFFSET 0
#endif
// Enable this to compile a read only version of spiffs.
// This will reduce binary size of spiffs. All code comprising modification
// of the file system will not be compiled. Some config will be ignored.
// HAL functions for erasing and writing to spi-flash may be null. Cache
// can be disabled for even further binary size reduction (and ram savings).
// Functions modifying the fs will return SPIFFS_ERR_RO_NOT_IMPL.
// If the file system cannot be mounted due to aborted erase operation and
// SPIFFS_USE_MAGIC is enabled, SPIFFS_ERR_RO_ABORTED_OPERATION will be
// returned.
// Might be useful for e.g. bootloaders and such.
#ifndef SPIFFS_READ_ONLY
#define SPIFFS_READ_ONLY 0
#endif
// Enable this to add a temporal file cache using the fd buffer.
// The effects of the cache is that SPIFFS_open will find the file faster in
// certain cases. It will make it a lot easier for spiffs to find files
// opened frequently, reducing number of readings from the spi flash for
// finding those files.
// This will grow each fd by 6 bytes. If your files are opened in patterns
// with a degree of temporal locality, the system is optimized.
// Examples can be letting spiffs serve web content, where one file is the css.
// The css is accessed for each html file that is opened, meaning it is
// accessed almost every second time a file is opened. Another example could be
// a log file that is often opened, written, and closed.
// The size of the cache is number of given file descriptors, as it piggybacks
// on the fd update mechanism. The cache lives in the closed file descriptors.
// When closed, the fd know the whereabouts of the file. Instead of forgetting
// this, the temporal cache will keep handling updates to that file even if the
// fd is closed. If the file is opened again, the location of the file is found
// directly. If all available descriptors become opened, all cache memory is
// lost.
#ifndef SPIFFS_TEMPORAL_FD_CACHE
#define SPIFFS_TEMPORAL_FD_CACHE 1
#endif
// Temporal file cache hit score. Each time a file is opened, all cached files
// will lose one point. If the opened file is found in cache, that entry will
// gain SPIFFS_TEMPORAL_CACHE_HIT_SCORE points. One can experiment with this
// value for the specific access patterns of the application. However, it must
// be between 1 (no gain for hitting a cached entry often) and 255.
#ifndef SPIFFS_TEMPORAL_CACHE_HIT_SCORE
#define SPIFFS_TEMPORAL_CACHE_HIT_SCORE 4
#endif
// Enable to be able to map object indices to memory.
// This allows for faster and more deterministic reading if cases of reading
// large files and when changing file offset by seeking around a lot.
// When mapping a file's index, the file system will be scanned for index pages
// and the info will be put in memory provided by user. When reading, the
// memory map can be looked up instead of searching for index pages on the
// medium. This way, user can trade memory against performance.
// Whole, parts of, or future parts not being written yet can be mapped. The
// memory array will be owned by spiffs and updated accordingly during garbage
// collecting or when modifying the indices. The latter is invoked by when the
// file is modified in some way. The index buffer is tied to the file
// descriptor.
#ifndef SPIFFS_IX_MAP
#define SPIFFS_IX_MAP 1
#endif
// By default SPIFFS in some cases relies on the property of NOR flash that bits
// cannot be set from 0 to 1 by writing and that controllers will ignore such
// bit changes. This results in fewer reads as SPIFFS can in some cases perform
// blind writes, with all bits set to 1 and only those it needs reset set to 0.
// Most of the chips and controllers allow this behavior, so the default is to
// use this technique. If your controller is one of the rare ones that don't,
// turn this option on and SPIFFS will perform a read-modify-write instead.
#ifndef SPIFFS_NO_BLIND_WRITES
#define SPIFFS_NO_BLIND_WRITES 0
#endif
// Set SPIFFS_TEST_VISUALISATION to non-zero to enable SPIFFS_vis function
// in the api. This function will visualize all filesystem using given printf
// function.
#ifndef SPIFFS_TEST_VISUALISATION
#define SPIFFS_TEST_VISUALISATION 1
#endif
#if SPIFFS_TEST_VISUALISATION
#ifndef spiffs_printf
#define spiffs_printf(...) printf(__VA_ARGS__)
#endif
// spiffs_printf argument for a free page
#ifndef SPIFFS_TEST_VIS_FREE_STR
#define SPIFFS_TEST_VIS_FREE_STR "_"
#endif
// spiffs_printf argument for a deleted page
#ifndef SPIFFS_TEST_VIS_DELE_STR
#define SPIFFS_TEST_VIS_DELE_STR "/"
#endif
// spiffs_printf argument for an index page for given object id
#ifndef SPIFFS_TEST_VIS_INDX_STR
#define SPIFFS_TEST_VIS_INDX_STR(id) "i"
#endif
// spiffs_printf argument for a data page for given object id
#ifndef SPIFFS_TEST_VIS_DATA_STR
#define SPIFFS_TEST_VIS_DATA_STR(id) "d"
#endif
#endif
// Types depending on configuration such as the amount of flash bytes
// given to spiffs file system in total (spiffs_file_system_size),
// the logical block size (log_block_size), and the logical page size
// (log_page_size)
// Block index type. Make sure the size of this type can hold
// the highest number of all blocks - i.e. spiffs_file_system_size / log_block_size
typedef u16_t spiffs_block_ix;
// Page index type. Make sure the size of this type can hold
// the highest page number of all pages - i.e. spiffs_file_system_size / log_page_size
typedef u16_t spiffs_page_ix;
// Object id type - most significant bit is reserved for index flag. Make sure the
// size of this type can hold the highest object id on a full system,
// i.e. 2 + (spiffs_file_system_size / (2*log_page_size))*2
typedef u16_t spiffs_obj_id;
// Object span index type. Make sure the size of this type can
// hold the largest possible span index on the system -
// i.e. (spiffs_file_system_size / log_page_size) - 1
typedef u16_t spiffs_span_ix;
#endif /* SPIFFS_CONFIG_H_ */

816
src/spiffs/spiffs.h
Unescape View File

@ -0,0 +1,816 @@
/*
* spiffs.h
*
* Created on: May 26, 2013
* Author: petera
*/
#ifndef SPIFFS_H_
#define SPIFFS_H_
#if defined(__cplusplus)
extern "C" {
#endif
#include "spiffs_config.h"
#define SPIFFS_OK 0
#define SPIFFS_ERR_NOT_MOUNTED -10000
#define SPIFFS_ERR_FULL -10001
#define SPIFFS_ERR_NOT_FOUND -10002
#define SPIFFS_ERR_END_OF_OBJECT -10003
#define SPIFFS_ERR_DELETED -10004
#define SPIFFS_ERR_NOT_FINALIZED -10005
#define SPIFFS_ERR_NOT_INDEX -10006
#define SPIFFS_ERR_OUT_OF_FILE_DESCS -10007
#define SPIFFS_ERR_FILE_CLOSED -10008
#define SPIFFS_ERR_FILE_DELETED -10009
#define SPIFFS_ERR_BAD_DESCRIPTOR -10010
#define SPIFFS_ERR_IS_INDEX -10011
#define SPIFFS_ERR_IS_FREE -10012
#define SPIFFS_ERR_INDEX_SPAN_MISMATCH -10013
#define SPIFFS_ERR_DATA_SPAN_MISMATCH -10014
#define SPIFFS_ERR_INDEX_REF_FREE -10015
#define SPIFFS_ERR_INDEX_REF_LU -10016
#define SPIFFS_ERR_INDEX_REF_INVALID -10017
#define SPIFFS_ERR_INDEX_FREE -10018
#define SPIFFS_ERR_INDEX_LU -10019
#define SPIFFS_ERR_INDEX_INVALID -10020
#define SPIFFS_ERR_NOT_WRITABLE -10021
#define SPIFFS_ERR_NOT_READABLE -10022
#define SPIFFS_ERR_CONFLICTING_NAME -10023
#define SPIFFS_ERR_NOT_CONFIGURED -10024
#define SPIFFS_ERR_NOT_A_FS -10025
#define SPIFFS_ERR_MOUNTED -10026
#define SPIFFS_ERR_ERASE_FAIL -10027
#define SPIFFS_ERR_MAGIC_NOT_POSSIBLE -10028
#define SPIFFS_ERR_NO_DELETED_BLOCKS -10029
#define SPIFFS_ERR_FILE_EXISTS -10030
#define SPIFFS_ERR_NOT_A_FILE -10031
#define SPIFFS_ERR_RO_NOT_IMPL -10032
#define SPIFFS_ERR_RO_ABORTED_OPERATION -10033
#define SPIFFS_ERR_PROBE_TOO_FEW_BLOCKS -10034
#define SPIFFS_ERR_PROBE_NOT_A_FS -10035
#define SPIFFS_ERR_NAME_TOO_LONG -10036
#define SPIFFS_ERR_IX_MAP_UNMAPPED -10037
#define SPIFFS_ERR_IX_MAP_MAPPED -10038
#define SPIFFS_ERR_IX_MAP_BAD_RANGE -10039
#define SPIFFS_ERR_SEEK_BOUNDS -10040
#define SPIFFS_ERR_INTERNAL -10050
#define SPIFFS_ERR_TEST -10100
// spiffs file descriptor index type. must be signed
typedef s16_t spiffs_file;
// spiffs file descriptor flags
typedef u16_t spiffs_flags;
// spiffs file mode
typedef u16_t spiffs_mode;
// object type
typedef u8_t spiffs_obj_type;
struct spiffs_t;
#if SPIFFS_HAL_CALLBACK_EXTRA
/* spi read call function type */
typedef s32_t (*spiffs_read)(struct spiffs_t *fs, u32_t addr, u32_t size, u8_t *dst);
/* spi write call function type */
typedef s32_t (*spiffs_write)(struct spiffs_t *fs, u32_t addr, u32_t size, u8_t *src);
/* spi erase call function type */
typedef s32_t (*spiffs_erase)(struct spiffs_t *fs, u32_t addr, u32_t size);
#else // SPIFFS_HAL_CALLBACK_EXTRA
/* spi read call function type */
typedef s32_t (*spiffs_read)(u32_t addr, u32_t size, u8_t *dst);
/* spi write call function type */
typedef s32_t (*spiffs_write)(u32_t addr, u32_t size, u8_t *src);
/* spi erase call function type */
typedef s32_t (*spiffs_erase)(u32_t addr, u32_t size);
#endif // SPIFFS_HAL_CALLBACK_EXTRA
/* file system check callback report operation */
typedef enum {
SPIFFS_CHECK_LOOKUP = 0,
SPIFFS_CHECK_INDEX,
SPIFFS_CHECK_PAGE
} spiffs_check_type;
/* file system check callback report type */
typedef enum {
SPIFFS_CHECK_PROGRESS = 0,
SPIFFS_CHECK_ERROR,
SPIFFS_CHECK_FIX_INDEX,
SPIFFS_CHECK_FIX_LOOKUP,
SPIFFS_CHECK_DELETE_ORPHANED_INDEX,
SPIFFS_CHECK_DELETE_PAGE,
SPIFFS_CHECK_DELETE_BAD_FILE
} spiffs_check_report;
/* file system check callback function */
#if SPIFFS_HAL_CALLBACK_EXTRA
typedef void (*spiffs_check_callback)(struct spiffs_t *fs, spiffs_check_type type, spiffs_check_report report,
u32_t arg1, u32_t arg2);
#else // SPIFFS_HAL_CALLBACK_EXTRA
typedef void (*spiffs_check_callback)(spiffs_check_type type, spiffs_check_report report,
u32_t arg1, u32_t arg2);
#endif // SPIFFS_HAL_CALLBACK_EXTRA
/* file system listener callback operation */
typedef enum {
/* the file has been created */
SPIFFS_CB_CREATED = 0,
/* the file has been updated or moved to another page */
SPIFFS_CB_UPDATED,
/* the file has been deleted */
SPIFFS_CB_DELETED
} spiffs_fileop_type;
/* file system listener callback function */
typedef void (*spiffs_file_callback)(struct spiffs_t *fs, spiffs_fileop_type op, spiffs_obj_id obj_id, spiffs_page_ix pix);
#ifndef SPIFFS_DBG
#define SPIFFS_DBG(...) \
printf(__VA_ARGS__)
#endif
#ifndef SPIFFS_GC_DBG
#define SPIFFS_GC_DBG(...) printf(__VA_ARGS__)
#endif
#ifndef SPIFFS_CACHE_DBG
#define SPIFFS_CACHE_DBG(...) printf(__VA_ARGS__)
#endif
#ifndef SPIFFS_CHECK_DBG
#define SPIFFS_CHECK_DBG(...) printf(__VA_ARGS__)
#endif
/* Any write to the filehandle is appended to end of the file */
#define SPIFFS_APPEND (1<<0)
#define SPIFFS_O_APPEND SPIFFS_APPEND
/* If the opened file exists, it will be truncated to zero length before opened */
#define SPIFFS_TRUNC (1<<1)
#define SPIFFS_O_TRUNC SPIFFS_TRUNC
/* If the opened file does not exist, it will be created before opened */
#define SPIFFS_CREAT (1<<2)
#define SPIFFS_O_CREAT SPIFFS_CREAT
/* The opened file may only be read */
#define SPIFFS_RDONLY (1<<3)
#define SPIFFS_O_RDONLY SPIFFS_RDONLY
/* The opened file may only be written */
#define SPIFFS_WRONLY (1<<4)
#define SPIFFS_O_WRONLY SPIFFS_WRONLY
/* The opened file may be both read and written */
#define SPIFFS_RDWR (SPIFFS_RDONLY | SPIFFS_WRONLY)
#define SPIFFS_O_RDWR SPIFFS_RDWR
/* Any writes to the filehandle will never be cached but flushed directly */
#define SPIFFS_DIRECT (1<<5)
#define SPIFFS_O_DIRECT SPIFFS_DIRECT
/* If SPIFFS_O_CREAT and SPIFFS_O_EXCL are set, SPIFFS_open() shall fail if the file exists */
#define SPIFFS_EXCL (1<<6)
#define SPIFFS_O_EXCL SPIFFS_EXCL
#define SPIFFS_SEEK_SET (0)
#define SPIFFS_SEEK_CUR (1)
#define SPIFFS_SEEK_END (2)
#define SPIFFS_TYPE_FILE (1)
#define SPIFFS_TYPE_DIR (2)
#define SPIFFS_TYPE_HARD_LINK (3)
#define SPIFFS_TYPE_SOFT_LINK (4)
#ifndef SPIFFS_LOCK
#define SPIFFS_LOCK(fs)
#endif
#ifndef SPIFFS_UNLOCK
#define SPIFFS_UNLOCK(fs)
#endif
// phys structs
// spiffs spi configuration struct
typedef struct {
// physical read function
spiffs_read hal_read_f;
// physical write function
spiffs_write hal_write_f;
// physical erase function
spiffs_erase hal_erase_f;
#if SPIFFS_SINGLETON == 0
// physical size of the spi flash
u32_t phys_size;
// physical offset in spi flash used for spiffs,
// must be on block boundary
u32_t phys_addr;
// physical size when erasing a block
u32_t phys_erase_block;
// logical size of a block, must be on physical
// block size boundary and must never be less than
// a physical block
u32_t log_block_size;
// logical size of a page, must be at least
// log_block_size / 8
u32_t log_page_size;
#endif
#if SPIFFS_FILEHDL_OFFSET
// an integer offset added to each file handle
u16_t fh_ix_offset;
#endif
} spiffs_config;
typedef struct spiffs_t {
// file system configuration
spiffs_config cfg;
// number of logical blocks
u32_t block_count;
// cursor for free blocks, block index
spiffs_block_ix free_cursor_block_ix;
// cursor for free blocks, entry index
int free_cursor_obj_lu_entry;
// cursor when searching, block index
spiffs_block_ix cursor_block_ix;
// cursor when searching, entry index
int cursor_obj_lu_entry;
// primary work buffer, size of a logical page
u8_t *lu_work;
// secondary work buffer, size of a logical page
u8_t *work;
// file descriptor memory area
u8_t *fd_space;
// available file descriptors
u32_t fd_count;
// last error
s32_t err_code;
// current number of free blocks
u32_t free_blocks;
// current number of busy pages
u32_t stats_p_allocated;
// current number of deleted pages
u32_t stats_p_deleted;
// flag indicating that garbage collector is cleaning
u8_t cleaning;
// max erase count amongst all blocks
spiffs_obj_id max_erase_count;
#if SPIFFS_GC_STATS
u32_t stats_gc_runs;
#endif
#if SPIFFS_CACHE
// cache memory
void *cache;
// cache size
u32_t cache_size;
#if SPIFFS_CACHE_STATS
u32_t cache_hits;
u32_t cache_misses;
#endif
#endif
// check callback function
spiffs_check_callback check_cb_f;
// file callback function
spiffs_file_callback file_cb_f;
// mounted flag
u8_t mounted;
// user data
void *user_data;
// config magic
u32_t config_magic;
} spiffs;
/* spiffs file status struct */
typedef struct {
spiffs_obj_id obj_id;
u32_t size;
spiffs_obj_type type;
spiffs_page_ix pix;
u8_t name[SPIFFS_OBJ_NAME_LEN];
#if SPIFFS_OBJ_META_LEN
u8_t meta[SPIFFS_OBJ_META_LEN];
#endif
} spiffs_stat;
struct spiffs_dirent {
spiffs_obj_id obj_id;
u8_t name[SPIFFS_OBJ_NAME_LEN];
spiffs_obj_type type;
u32_t size;
spiffs_page_ix pix;
#if SPIFFS_OBJ_META_LEN
u8_t meta[SPIFFS_OBJ_META_LEN];
#endif
};
typedef struct {
spiffs *fs;
spiffs_block_ix block;
int entry;
} spiffs_DIR;
#if SPIFFS_IX_MAP
typedef struct {
// buffer with looked up data pixes
spiffs_page_ix *map_buf;
// precise file byte offset
u32_t offset;
// start data span index of lookup buffer
spiffs_span_ix start_spix;
// end data span index of lookup buffer
spiffs_span_ix end_spix;
} spiffs_ix_map;
#endif
// functions
#if SPIFFS_USE_MAGIC && SPIFFS_USE_MAGIC_LENGTH && SPIFFS_SINGLETON==0
/**
* Special function. This takes a spiffs config struct and returns the number
* of blocks this file system was formatted with. This function relies on
* that following info is set correctly in given config struct:
*
* phys_addr, log_page_size, and log_block_size.
*
* Also, hal_read_f must be set in the config struct.
*
* One must be sure of the correct page size and that the physical address is
* correct in the probed file system when calling this function. It is not
* checked if the phys_addr actually points to the start of the file system,
* so one might get a false positive if entering a phys_addr somewhere in the
* middle of the file system at block boundary. In addition, it is not checked
* if the page size is actually correct. If it is not, weird file system sizes
* will be returned.
*
* If this function detects a file system it returns the assumed file system
* size, which can be used to set the phys_size.
*
* Otherwise, it returns an error indicating why it is not regarded as a file
* system.
*
* Note: this function is not protected with SPIFFS_LOCK and SPIFFS_UNLOCK
* macros. It returns the error code directly, instead of as read by
* SPIFFS_errno.
*
* @param config essential parts of the physical and logical
* configuration of the file system.
*/
s32_t SPIFFS_probe_fs(spiffs_config *config);
#endif // SPIFFS_USE_MAGIC && SPIFFS_USE_MAGIC_LENGTH && SPIFFS_SINGLETON==0
/**
* Initializes the file system dynamic parameters and mounts the filesystem.
* If SPIFFS_USE_MAGIC is enabled the mounting may fail with SPIFFS_ERR_NOT_A_FS
* if the flash does not contain a recognizable file system.
* In this case, SPIFFS_format must be called prior to remounting.
* @param fs the file system struct
* @param config the physical and logical configuration of the file system
* @param work a memory work buffer comprising 2*config->log_page_size
* bytes used throughout all file system operations
* @param fd_space memory for file descriptors
* @param fd_space_size memory size of file descriptors
* @param cache memory for cache, may be null
* @param cache_size memory size of cache
* @param check_cb_f callback function for reporting during consistency checks
*/
s32_t SPIFFS_mount(spiffs *fs, spiffs_config *config, u8_t *work,
u8_t *fd_space, u32_t fd_space_size,
void *cache, u32_t cache_size,
spiffs_check_callback check_cb_f);
/**
* Unmounts the file system. All file handles will be flushed of any
* cached writes and closed.
* @param fs the file system struct
*/
void SPIFFS_unmount(spiffs *fs);
/**
* Creates a new file.
* @param fs the file system struct
* @param path the path of the new file
* @param mode ignored, for posix compliance
*/
s32_t SPIFFS_creat(spiffs *fs, const char *path, spiffs_mode mode);
/**
* Opens/creates a file.
* @param fs the file system struct
* @param path the path of the new file
* @param flags the flags for the open command, can be combinations of
* SPIFFS_O_APPEND, SPIFFS_O_TRUNC, SPIFFS_O_CREAT, SPIFFS_O_RDONLY,
* SPIFFS_O_WRONLY, SPIFFS_O_RDWR, SPIFFS_O_DIRECT, SPIFFS_O_EXCL
* @param mode ignored, for posix compliance
*/
spiffs_file SPIFFS_open(spiffs *fs, const char *path, spiffs_flags flags, spiffs_mode mode);
/**
* Opens a file by given dir entry.
* Optimization purposes, when traversing a file system with SPIFFS_readdir
* a normal SPIFFS_open would need to traverse the filesystem again to find
* the file, whilst SPIFFS_open_by_dirent already knows where the file resides.
* @param fs the file system struct
* @param e the dir entry to the file
* @param flags the flags for the open command, can be combinations of
* SPIFFS_APPEND, SPIFFS_TRUNC, SPIFFS_CREAT, SPIFFS_RD_ONLY,
* SPIFFS_WR_ONLY, SPIFFS_RDWR, SPIFFS_DIRECT.
* SPIFFS_CREAT will have no effect in this case.
* @param mode ignored, for posix compliance
*/
spiffs_file SPIFFS_open_by_dirent(spiffs *fs, struct spiffs_dirent *e, spiffs_flags flags, spiffs_mode mode);
/**
* Opens a file by given page index.
* Optimization purposes, opens a file by directly pointing to the page
* index in the spi flash.
* If the page index does not point to a file header SPIFFS_ERR_NOT_A_FILE
* is returned.
* @param fs the file system struct
* @param page_ix the page index
* @param flags the flags for the open command, can be combinations of
* SPIFFS_APPEND, SPIFFS_TRUNC, SPIFFS_CREAT, SPIFFS_RD_ONLY,
* SPIFFS_WR_ONLY, SPIFFS_RDWR, SPIFFS_DIRECT.
* SPIFFS_CREAT will have no effect in this case.
* @param mode ignored, for posix compliance
*/
spiffs_file SPIFFS_open_by_page(spiffs *fs, spiffs_page_ix page_ix, spiffs_flags flags, spiffs_mode mode);
/**
* Reads from given filehandle.
* @param fs the file system struct
* @param fh the filehandle
* @param buf where to put read data
* @param len how much to read
* @returns number of bytes read, or -1 if error
*/
s32_t SPIFFS_read(spiffs *fs, spiffs_file fh, void *buf, s32_t len);
/**
* Writes to given filehandle.
* @param fs the file system struct
* @param fh the filehandle
* @param buf the data to write
* @param len how much to write
* @returns number of bytes written, or -1 if error
*/
s32_t SPIFFS_write(spiffs *fs, spiffs_file fh, void *buf, s32_t len);
/**
* Moves the read/write file offset. Resulting offset is returned or negative if error.
* lseek(fs, fd, 0, SPIFFS_SEEK_CUR) will thus return current offset.
* @param fs the file system struct
* @param fh the filehandle
* @param offs how much/where to move the offset
* @param whence if SPIFFS_SEEK_SET, the file offset shall be set to offset bytes
* if SPIFFS_SEEK_CUR, the file offset shall be set to its current location plus offset
* if SPIFFS_SEEK_END, the file offset shall be set to the size of the file plus offse, which should be negative
*/
s32_t SPIFFS_lseek(spiffs *fs, spiffs_file fh, s32_t offs, int whence);
/**
* Removes a file by path
* @param fs the file system struct
* @param path the path of the file to remove
*/
s32_t SPIFFS_remove(spiffs *fs, const char *path);
/**
* Removes a file by filehandle
* @param fs the file system struct
* @param fh the filehandle of the file to remove
*/
s32_t SPIFFS_fremove(spiffs *fs, spiffs_file fh);
/**
* Gets file status by path
* @param fs the file system struct
* @param path the path of the file to stat
* @param s the stat struct to populate
*/
s32_t SPIFFS_stat(spiffs *fs, const char *path, spiffs_stat *s);
/**
* Gets file status by filehandle
* @param fs the file system struct
* @param fh the filehandle of the file to stat
* @param s the stat struct to populate
*/
s32_t SPIFFS_fstat(spiffs *fs, spiffs_file fh, spiffs_stat *s);
/**
* Flushes all pending write operations from cache for given file
* @param fs the file system struct
* @param fh the filehandle of the file to flush
*/
s32_t SPIFFS_fflush(spiffs *fs, spiffs_file fh);
/**
* Closes a filehandle. If there are pending write operations, these are finalized before closing.
* @param fs the file system struct
* @param fh the filehandle of the file to close
*/
s32_t SPIFFS_close(spiffs *fs, spiffs_file fh);
/**
* Renames a file
* @param fs the file system struct
* @param old path of file to rename
* @param newPath new path of file
*/
s32_t SPIFFS_rename(spiffs *fs, const char *old, const char *newPath);
#if SPIFFS_OBJ_META_LEN
/**
* Updates file's metadata
* @param fs the file system struct
* @param path path to the file
* @param meta new metadata. must be SPIFFS_OBJ_META_LEN bytes long.
*/
s32_t SPIFFS_update_meta(spiffs *fs, const char *name, const void *meta);
/**
* Updates file's metadata
* @param fs the file system struct
* @param fh file handle of the file
* @param meta new metadata. must be SPIFFS_OBJ_META_LEN bytes long.
*/
s32_t SPIFFS_fupdate_meta(spiffs *fs, spiffs_file fh, const void *meta);
#endif
/**
* Returns last error of last file operation.
* @param fs the file system struct
*/
s32_t SPIFFS_errno(spiffs *fs);
/**
* Clears last error.
* @param fs the file system struct
*/
void SPIFFS_clearerr(spiffs *fs);
/**
* Opens a directory stream corresponding to the given name.
* The stream is positioned at the first entry in the directory.
* On hydrogen builds the name argument is ignored as hydrogen builds always correspond
* to a flat file structure - no directories.
* @param fs the file system struct
* @param name the name of the directory
* @param d pointer the directory stream to be populated
*/
spiffs_DIR *SPIFFS_opendir(spiffs *fs, const char *name, spiffs_DIR *d);
/**
* Closes a directory stream
* @param d the directory stream to close
*/
s32_t SPIFFS_closedir(spiffs_DIR *d);
/**
* Reads a directory into given spifs_dirent struct.
* @param d pointer to the directory stream
* @param e the dirent struct to be populated
* @returns null if error or end of stream, else given dirent is returned
*/
struct spiffs_dirent *SPIFFS_readdir(spiffs_DIR *d, struct spiffs_dirent *e);
/**
* Runs a consistency check on given filesystem.
* @param fs the file system struct
*/
s32_t SPIFFS_check(spiffs *fs);
/**
* Returns number of total bytes available and number of used bytes.
* This is an estimation, and depends on if there a many files with little
* data or few files with much data.
* NB: If used number of bytes exceeds total bytes, a SPIFFS_check should
* run. This indicates a power loss in midst of things. In worst case
* (repeated powerlosses in mending or gc) you might have to delete some files.
*
* @param fs the file system struct
* @param total total number of bytes in filesystem
* @param used used number of bytes in filesystem
*/
s32_t SPIFFS_info(spiffs *fs, u32_t *total, u32_t *used);
/**
* Formats the entire file system. All data will be lost.
* The filesystem must not be mounted when calling this.
*
* NB: formatting is awkward. Due to backwards compatibility, SPIFFS_mount
* MUST be called prior to formatting in order to configure the filesystem.
* If SPIFFS_mount succeeds, SPIFFS_unmount must be called before calling
* SPIFFS_format.
* If SPIFFS_mount fails, SPIFFS_format can be called directly without calling
* SPIFFS_unmount first.
*
* @param fs the file system struct
*/
s32_t SPIFFS_format(spiffs *fs);
/**
* Returns nonzero if spiffs is mounted, or zero if unmounted.
* @param fs the file system struct
*/
u8_t SPIFFS_mounted(spiffs *fs);
/**
* Tries to find a block where most or all pages are deleted, and erase that
* block if found. Does not care for wear levelling. Will not move pages
* around.
* If parameter max_free_pages are set to 0, only blocks with only deleted
* pages will be selected.
*
* NB: the garbage collector is automatically called when spiffs needs free
* pages. The reason for this function is to give possibility to do background
* tidying when user knows the system is idle.
*
* Use with care.
*
* Setting max_free_pages to anything larger than zero will eventually wear
* flash more as a block containing free pages can be erased.
*
* Will set err_no to SPIFFS_OK if a block was found and erased,
* SPIFFS_ERR_NO_DELETED_BLOCK if no matching block was found,
* or other error.
*
* @param fs the file system struct
* @param max_free_pages maximum number allowed free pages in block
*/
s32_t SPIFFS_gc_quick(spiffs *fs, u16_t max_free_pages);
/**
* Will try to make room for given amount of bytes in the filesystem by moving
* pages and erasing blocks.
* If it is physically impossible, err_no will be set to SPIFFS_ERR_FULL. If
* there already is this amount (or more) of free space, SPIFFS_gc will
* silently return. It is recommended to call SPIFFS_info before invoking
* this method in order to determine what amount of bytes to give.
*
* NB: the garbage collector is automatically called when spiffs needs free
* pages. The reason for this function is to give possibility to do background
* tidying when user knows the system is idle.
*
* Use with care.
*
* @param fs the file system struct
* @param size amount of bytes that should be freed
*/
s32_t SPIFFS_gc(spiffs *fs, u32_t size);
/**
* Check if EOF reached.
* @param fs the file system struct
* @param fh the filehandle of the file to check
*/
s32_t SPIFFS_eof(spiffs *fs, spiffs_file fh);
/**
* Get position in file.
* @param fs the file system struct
* @param fh the filehandle of the file to check
*/
s32_t SPIFFS_tell(spiffs *fs, spiffs_file fh);
/**
* Registers a callback function that keeps track on operations on file
* headers. Do note, that this callback is called from within internal spiffs
* mechanisms. Any operations on the actual file system being callbacked from
* in this callback will mess things up for sure - do not do this.
* This can be used to track where files are and move around during garbage
* collection, which in turn can be used to build location tables in ram.
* Used in conjuction with SPIFFS_open_by_page this may improve performance
* when opening a lot of files.
* Must be invoked after mount.
*
* @param fs the file system struct
* @param cb_func the callback on file operations
*/
s32_t SPIFFS_set_file_callback_func(spiffs *fs, spiffs_file_callback cb_func);
#if SPIFFS_IX_MAP
/**
* Maps the first level index lookup to a given memory map.
* This will make reading big files faster, as the memory map will be used for
* looking up data pages instead of searching for the indices on the physical
* medium. When mapping, all affected indicies are found and the information is
* copied to the array.
* Whole file or only parts of it may be mapped. The index map will cover file
* contents from argument offset until and including arguments (offset+len).
* It is valid to map a longer range than the current file size. The map will
* then be populated when the file grows.
* On garbage collections and file data page movements, the map array will be
* automatically updated. Do not tamper with the map array, as this contains
* the references to the data pages. Modifying it from outside will corrupt any
* future readings using this file descriptor.
* The map will no longer be used when the file descriptor closed or the file
* is unmapped.
* This can be useful to get faster and more deterministic timing when reading
* large files, or when seeking and reading a lot within a file.
* @param fs the file system struct
* @param fh the file handle of the file to map
* @param map a spiffs_ix_map struct, describing the index map
* @param offset absolute file offset where to start the index map
* @param len length of the mapping in actual file bytes
* @param map_buf the array buffer for the look up data - number of required
* elements in the array can be derived from function
* SPIFFS_bytes_to_ix_map_entries given the length
*/
s32_t SPIFFS_ix_map(spiffs *fs, spiffs_file fh, spiffs_ix_map *map,
u32_t offset, u32_t len, spiffs_page_ix *map_buf);
/**
* Unmaps the index lookup from this filehandle. All future readings will
* proceed as normal, requiring reading of the first level indices from
* physical media.
* The map and map buffer given in function SPIFFS_ix_map will no longer be
* referenced by spiffs.
* It is not strictly necessary to unmap a file before closing it, as closing
* a file will automatically unmap it.
* @param fs the file system struct
* @param fh the file handle of the file to unmap
*/
s32_t SPIFFS_ix_unmap(spiffs *fs, spiffs_file fh);
/**
* Moves the offset for the index map given in function SPIFFS_ix_map. Parts or
* all of the map buffer will repopulated.
* @param fs the file system struct
* @param fh the mapped file handle of the file to remap
* @param offset new absolute file offset where to start the index map
*/
s32_t SPIFFS_ix_remap(spiffs *fs, spiffs_file fh, u32_t offs);
/**
* Utility function to get number of spiffs_page_ix entries a map buffer must
* contain on order to map given amount of file data in bytes.
* See function SPIFFS_ix_map and SPIFFS_ix_map_entries_to_bytes.
* @param fs the file system struct
* @param bytes number of file data bytes to map
* @return needed number of elements in a spiffs_page_ix array needed to
* map given amount of bytes in a file
*/
s32_t SPIFFS_bytes_to_ix_map_entries(spiffs *fs, u32_t bytes);
/**
* Utility function to amount of file data bytes that can be mapped when
* mapping a file with buffer having given number of spiffs_page_ix entries.
* See function SPIFFS_ix_map and SPIFFS_bytes_to_ix_map_entries.
* @param fs the file system struct
* @param map_page_ix_entries number of entries in a spiffs_page_ix array
* @return amount of file data in bytes that can be mapped given a map
* buffer having given amount of spiffs_page_ix entries
*/
s32_t SPIFFS_ix_map_entries_to_bytes(spiffs *fs, u32_t map_page_ix_entries);
#endif // SPIFFS_IX_MAP
#if SPIFFS_TEST_VISUALISATION
/**
* Prints out a visualization of the filesystem.
* @param fs the file system struct
*/
s32_t SPIFFS_vis(spiffs *fs);
#endif
#if SPIFFS_BUFFER_HELP
/**
* Returns number of bytes needed for the filedescriptor buffer given
* amount of file descriptors.
*/
u32_t SPIFFS_buffer_bytes_for_filedescs(spiffs *fs, u32_t num_descs);
#if SPIFFS_CACHE
/**
* Returns number of bytes needed for the cache buffer given
* amount of cache pages.
*/
u32_t SPIFFS_buffer_bytes_for_cache(spiffs *fs, u32_t num_pages);
#endif
#endif
#if SPIFFS_CACHE
#endif
#if defined(__cplusplus)
}
#endif
#endif /* SPIFFS_H_ */

319
src/spiffs/spiffs_cache.c
Unescape View File

@ -0,0 +1,319 @@
/*
* spiffs_cache.c
*
* Created on: Jun 23, 2013
* Author: petera
*/
#include "spiffs.h"
#include "spiffs_nucleus.h"
#if SPIFFS_CACHE
// returns cached page for give page index, or null if no such cached page
static spiffs_cache_page *spiffs_cache_page_get(spiffs *fs, spiffs_page_ix pix) {
spiffs_cache *cache = spiffs_get_cache(fs);
if ((cache->cpage_use_map & cache->cpage_use_mask) == 0) return 0;
int i;
for (i = 0; i < cache->cpage_count; i++) {
spiffs_cache_page *cp = spiffs_get_cache_page_hdr(fs, cache, i);
if ((cache->cpage_use_map & (1<<i)) &&
(cp->flags & SPIFFS_CACHE_FLAG_TYPE_WR) == 0 &&
cp->pix == pix ) {
//SPIFFS_CACHE_DBG("CACHE_GET: have cache page "_SPIPRIi" for "_SPIPRIpg"\n", i, pix);
cp->last_access = cache->last_access;
return cp;
}
}
//SPIFFS_CACHE_DBG("CACHE_GET: no cache for "_SPIPRIpg"\n", pix);
return 0;
}
// frees cached page
static s32_t spiffs_cache_page_free(spiffs *fs, int ix, u8_t write_back) {
s32_t res = SPIFFS_OK;
spiffs_cache *cache = spiffs_get_cache(fs);
spiffs_cache_page *cp = spiffs_get_cache_page_hdr(fs, cache, ix);
if (cache->cpage_use_map & (1<<ix)) {
if (write_back &&
(cp->flags & SPIFFS_CACHE_FLAG_TYPE_WR) == 0 &&
(cp->flags & SPIFFS_CACHE_FLAG_DIRTY)) {
u8_t *mem = spiffs_get_cache_page(fs, cache, ix);
SPIFFS_CACHE_DBG("CACHE_FREE: write cache page "_SPIPRIi" pix "_SPIPRIpg"\n", ix, cp->pix);
res = SPIFFS_HAL_WRITE(fs, SPIFFS_PAGE_TO_PADDR(fs, cp->pix), SPIFFS_CFG_LOG_PAGE_SZ(fs), mem);
}
#if SPIFFS_CACHE_WR
if (cp->flags & SPIFFS_CACHE_FLAG_TYPE_WR) {
SPIFFS_CACHE_DBG("CACHE_FREE: free cache page "_SPIPRIi" objid "_SPIPRIid"\n", ix, cp->obj_id);
} else
#endif
{
SPIFFS_CACHE_DBG("CACHE_FREE: free cache page "_SPIPRIi" pix "_SPIPRIpg"\n", ix, cp->pix);
}
cache->cpage_use_map &= ~(1 << ix);
cp->flags = 0;
}
return res;
}
// removes the oldest accessed cached page
static s32_t spiffs_cache_page_remove_oldest(spiffs *fs, u8_t flag_mask, u8_t flags) {
s32_t res = SPIFFS_OK;
spiffs_cache *cache = spiffs_get_cache(fs);
if ((cache->cpage_use_map & cache->cpage_use_mask) != cache->cpage_use_mask) {
// at least one free cpage
return SPIFFS_OK;
}
// all busy, scan thru all to find the cpage which has oldest access
int i;
int cand_ix = -1;
u32_t oldest_val = 0;
for (i = 0; i < cache->cpage_count; i++) {
spiffs_cache_page *cp = spiffs_get_cache_page_hdr(fs, cache, i);
if ((cache->last_access - cp->last_access) > oldest_val &&
(cp->flags & flag_mask) == flags) {
oldest_val = cache->last_access - cp->last_access;
cand_ix = i;
}
}
if (cand_ix >= 0) {
res = spiffs_cache_page_free(fs, cand_ix, 1);
}
return res;
}
// allocates a new cached page and returns it, or null if all cache pages are busy
static spiffs_cache_page *spiffs_cache_page_allocate(spiffs *fs) {
spiffs_cache *cache = spiffs_get_cache(fs);
if (cache->cpage_use_map == 0xffffffff) {
// out of cache memory
return 0;
}
int i;
for (i = 0; i < cache->cpage_count; i++) {
if ((cache->cpage_use_map & (1<<i)) == 0) {
spiffs_cache_page *cp = spiffs_get_cache_page_hdr(fs, cache, i);
cache->cpage_use_map |= (1<<i);
cp->last_access = cache->last_access;
//SPIFFS_CACHE_DBG("CACHE_ALLO: allocated cache page "_SPIPRIi"\n", i);
return cp;
}
}
// out of cache entries
return 0;
}
// drops the cache page for give page index
void spiffs_cache_drop_page(spiffs *fs, spiffs_page_ix pix) {
spiffs_cache_page *cp = spiffs_cache_page_get(fs, pix);
if (cp) {
spiffs_cache_page_free(fs, cp->ix, 0);
}
}
// ------------------------------
// reads from spi flash or the cache
s32_t spiffs_phys_rd(
spiffs *fs,
u8_t op,
spiffs_file fh,
u32_t addr,
u32_t len,
u8_t *dst) {
(void)fh;
s32_t res = SPIFFS_OK;
spiffs_cache *cache = spiffs_get_cache(fs);
spiffs_cache_page *cp = spiffs_cache_page_get(fs, SPIFFS_PADDR_TO_PAGE(fs, addr));
cache->last_access++;
if (cp) {
// we've already got one, you see
#if SPIFFS_CACHE_STATS
fs->cache_hits++;
#endif
cp->last_access = cache->last_access;
u8_t *mem = spiffs_get_cache_page(fs, cache, cp->ix);
_SPIFFS_MEMCPY(dst, &mem[SPIFFS_PADDR_TO_PAGE_OFFSET(fs, addr)], len);
} else {
if ((op & SPIFFS_OP_TYPE_MASK) == SPIFFS_OP_T_OBJ_LU2) {
// for second layer lookup functions, we do not cache in order to prevent shredding
return SPIFFS_HAL_READ(fs, addr, len, dst);
}
#if SPIFFS_CACHE_STATS
fs->cache_misses++;
#endif
// this operation will always free one cache page (unless all already free),
// the result code stems from the write operation of the possibly freed cache page
res = spiffs_cache_page_remove_oldest(fs, SPIFFS_CACHE_FLAG_TYPE_WR, 0);
cp = spiffs_cache_page_allocate(fs);
if (cp) {
cp->flags = SPIFFS_CACHE_FLAG_WRTHRU;
cp->pix = SPIFFS_PADDR_TO_PAGE(fs, addr);
SPIFFS_CACHE_DBG("CACHE_ALLO: allocated cache page "_SPIPRIi" for pix "_SPIPRIpg "\n", cp->ix, cp->pix);
s32_t res2 = SPIFFS_HAL_READ(fs,
addr - SPIFFS_PADDR_TO_PAGE_OFFSET(fs, addr),
SPIFFS_CFG_LOG_PAGE_SZ(fs),
spiffs_get_cache_page(fs, cache, cp->ix));
if (res2 != SPIFFS_OK) {
// honor read failure before possible write failure (bad idea?)
res = res2;
}
u8_t *mem = spiffs_get_cache_page(fs, cache, cp->ix);
_SPIFFS_MEMCPY(dst, &mem[SPIFFS_PADDR_TO_PAGE_OFFSET(fs, addr)], len);
} else {
// this will never happen, last resort for sake of symmetry
s32_t res2 = SPIFFS_HAL_READ(fs, addr, len, dst);
if (res2 != SPIFFS_OK) {
// honor read failure before possible write failure (bad idea?)
res = res2;
}
}
}
return res;
}
// writes to spi flash and/or the cache
s32_t spiffs_phys_wr(
spiffs *fs,
u8_t op,
spiffs_file fh,
u32_t addr,
u32_t len,
u8_t *src) {
(void)fh;
spiffs_page_ix pix = SPIFFS_PADDR_TO_PAGE(fs, addr);
spiffs_cache *cache = spiffs_get_cache(fs);
spiffs_cache_page *cp = spiffs_cache_page_get(fs, pix);
if (cp && (op & SPIFFS_OP_COM_MASK) != SPIFFS_OP_C_WRTHRU) {
// have a cache page
// copy in data to cache page
if ((op & SPIFFS_OP_COM_MASK) == SPIFFS_OP_C_DELE &&
(op & SPIFFS_OP_TYPE_MASK) != SPIFFS_OP_T_OBJ_LU) {
// page is being deleted, wipe from cache - unless it is a lookup page
spiffs_cache_page_free(fs, cp->ix, 0);
return SPIFFS_HAL_WRITE(fs, addr, len, src);
}
u8_t *mem = spiffs_get_cache_page(fs, cache, cp->ix);
_SPIFFS_MEMCPY(&mem[SPIFFS_PADDR_TO_PAGE_OFFSET(fs, addr)], src, len);
cache->last_access++;
cp->last_access = cache->last_access;
if (cp->flags & SPIFFS_CACHE_FLAG_WRTHRU) {
// page is being updated, no write-cache, just pass thru
return SPIFFS_HAL_WRITE(fs, addr, len, src);
} else {
return SPIFFS_OK;
}
} else {
// no cache page, no write cache - just write thru
return SPIFFS_HAL_WRITE(fs, addr, len, src);
}
}
#if SPIFFS_CACHE_WR
// returns the cache page that this fd refers, or null if no cache page
spiffs_cache_page *spiffs_cache_page_get_by_fd(spiffs *fs, spiffs_fd *fd) {
spiffs_cache *cache = spiffs_get_cache(fs);
if ((cache->cpage_use_map & cache->cpage_use_mask) == 0) {
// all cpages free, no cpage cannot be assigned to obj_id
return 0;
}
int i;
for (i = 0; i < cache->cpage_count; i++) {
spiffs_cache_page *cp = spiffs_get_cache_page_hdr(fs, cache, i);
if ((cache->cpage_use_map & (1<<i)) &&
(cp->flags & SPIFFS_CACHE_FLAG_TYPE_WR) &&
cp->obj_id == fd->obj_id) {
return cp;
}
}
return 0;
}
// allocates a new cache page and refers this to given fd - flushes an old cache
// page if all cache is busy
spiffs_cache_page *spiffs_cache_page_allocate_by_fd(spiffs *fs, spiffs_fd *fd) {
// before this function is called, it is ensured that there is no already existing
// cache page with same object id
spiffs_cache_page_remove_oldest(fs, SPIFFS_CACHE_FLAG_TYPE_WR, 0);
spiffs_cache_page *cp = spiffs_cache_page_allocate(fs);
if (cp == 0) {
// could not get cache page
return 0;
}
cp->flags = SPIFFS_CACHE_FLAG_TYPE_WR;
cp->obj_id = fd->obj_id;
fd->cache_page = cp;
SPIFFS_CACHE_DBG("CACHE_ALLO: allocated cache page "_SPIPRIi" for fd "_SPIPRIfd ":"_SPIPRIid "\n", cp->ix, fd->file_nbr, fd->obj_id);
return cp;
}
// unrefers all fds that this cache page refers to and releases the cache page
void spiffs_cache_fd_release(spiffs *fs, spiffs_cache_page *cp) {
if (cp == 0) return;
u32_t i;
spiffs_fd *fds = (spiffs_fd *)fs->fd_space;
for (i = 0; i < fs->fd_count; i++) {
spiffs_fd *cur_fd = &fds[i];
if (cur_fd->file_nbr != 0 && cur_fd->cache_page == cp) {
cur_fd->cache_page = 0;
}
}
spiffs_cache_page_free(fs, cp->ix, 0);
cp->obj_id = 0;
}
#endif
// initializes the cache
void spiffs_cache_init(spiffs *fs) {
if (fs->cache == 0) return;
u32_t sz = fs->cache_size;
u32_t cache_mask = 0;
int i;
int cache_entries =
(sz - sizeof(spiffs_cache)) / (SPIFFS_CACHE_PAGE_SIZE(fs));
if (cache_entries <= 0) return;
for (i = 0; i < cache_entries; i++) {
cache_mask <<= 1;
cache_mask |= 1;
}
spiffs_cache cache;
memset(&cache, 0, sizeof(spiffs_cache));
cache.cpage_count = cache_entries;
cache.cpages = (u8_t *)((u8_t *)fs->cache + sizeof(spiffs_cache));
cache.cpage_use_map = 0xffffffff;
cache.cpage_use_mask = cache_mask;
_SPIFFS_MEMCPY(fs->cache, &cache, sizeof(spiffs_cache));
spiffs_cache *c = spiffs_get_cache(fs);
memset(c->cpages, 0, c->cpage_count * SPIFFS_CACHE_PAGE_SIZE(fs));
c->cpage_use_map &= ~(c->cpage_use_mask);
for (i = 0; i < cache.cpage_count; i++) {
spiffs_get_cache_page_hdr(fs, c, i)->ix = i;
}
}
#endif // SPIFFS_CACHE

1008
src/spiffs/spiffs_check.c
View File

File diff suppressed because it is too large Load Diff

369
src/spiffs/spiffs_config.h
Unescape View File

@ -0,0 +1,369 @@
/*
* spiffs_config.h
*
* Created on: Jul 3, 2013
* Author: petera
*/
#ifndef SPIFFS_CONFIG_H_
#define SPIFFS_CONFIG_H_
#include <esp8266.h>
/* type definitions used in spiffs code */
typedef int32_t s32_t;
typedef uint32_t u32_t;
typedef int16_t s16_t;
typedef uint16_t u16_t;
typedef int8_t s8_t;
typedef uint8_t u8_t;
// compile time switches
// Set generic spiffs debug output call.
#ifndef SPIFFS_DBG
#define SPIFFS_DBG(_f, ...) //printf(_f, ## __VA_ARGS__)
#endif
// Set spiffs debug output call for garbage collecting.
#ifndef SPIFFS_GC_DBG
#define SPIFFS_GC_DBG(_f, ...) //printf(_f, ## __VA_ARGS__)
#endif
// Set spiffs debug output call for caching.
#ifndef SPIFFS_CACHE_DBG
#define SPIFFS_CACHE_DBG(_f, ...) //printf(_f, ## __VA_ARGS__)
#endif
// Set spiffs debug output call for system consistency checks.
#ifndef SPIFFS_CHECK_DBG
#define SPIFFS_CHECK_DBG(_f, ...) //printf(_f, ## __VA_ARGS__)
#endif
// Set spiffs debug output call for all api invocations.
#ifndef SPIFFS_API_DBG
#define SPIFFS_API_DBG(_f, ...) //printf(_f, ## __VA_ARGS__)
#endif
// Defines spiffs debug print formatters
// some general signed number
#ifndef _SPIPRIi
#define _SPIPRIi "%d"
#endif
// address
#ifndef _SPIPRIad
#define _SPIPRIad "%08x"
#endif
// block
#ifndef _SPIPRIbl
#define _SPIPRIbl "%04x"
#endif
// page
#ifndef _SPIPRIpg
#define _SPIPRIpg "%04x"
#endif
// span index
#ifndef _SPIPRIsp
#define _SPIPRIsp "%04x"
#endif
// file descriptor
#ifndef _SPIPRIfd
#define _SPIPRIfd "%d"
#endif
// file object id
#ifndef _SPIPRIid
#define _SPIPRIid "%04x"
#endif
// file flags
#ifndef _SPIPRIfl
#define _SPIPRIfl "%02x"
#endif
// Enable/disable API functions to determine exact number of bytes
// for filedescriptor and cache buffers. Once decided for a configuration,
// this can be disabled to reduce flash.
#ifndef SPIFFS_BUFFER_HELP
#define SPIFFS_BUFFER_HELP 0
#endif
// Enables/disable memory read caching of nucleus file system operations.
// If enabled, memory area must be provided for cache in SPIFFS_mount.
#ifndef SPIFFS_CACHE
#define SPIFFS_CACHE 1
#endif
#if SPIFFS_CACHE
// Enables memory write caching for file descriptors in hydrogen
#ifndef SPIFFS_CACHE_WR
#define SPIFFS_CACHE_WR 1
#endif
// Enable/disable statistics on caching. Debug/test purpose only.
#ifndef SPIFFS_CACHE_STATS
#define SPIFFS_CACHE_STATS 1
#endif
#endif
// Always check header of each accessed page to ensure consistent state.
// If enabled it will increase number of reads, will increase flash.
#ifndef SPIFFS_PAGE_CHECK
#define SPIFFS_PAGE_CHECK 1
#endif
// Define maximum number of gc runs to perform to reach desired free pages.
#ifndef SPIFFS_GC_MAX_RUNS
#define SPIFFS_GC_MAX_RUNS 5
#endif
// Enable/disable statistics on gc. Debug/test purpose only.
#ifndef SPIFFS_GC_STATS
#define SPIFFS_GC_STATS 1
#endif
// Garbage collecting examines all pages in a block which and sums up
// to a block score. Deleted pages normally gives positive score and
// used pages normally gives a negative score (as these must be moved).
// To have a fair wear-leveling, the erase age is also included in score,
// whose factor normally is the most positive.
// The larger the score, the more likely it is that the block will
// picked for garbage collection.
// Garbage collecting heuristics - weight used for deleted pages.
#ifndef SPIFFS_GC_HEUR_W_DELET
#define SPIFFS_GC_HEUR_W_DELET (5)
#endif
// Garbage collecting heuristics - weight used for used pages.
#ifndef SPIFFS_GC_HEUR_W_USED
#define SPIFFS_GC_HEUR_W_USED (-1)
#endif
// Garbage collecting heuristics - weight used for time between
// last erased and erase of this block.
#ifndef SPIFFS_GC_HEUR_W_ERASE_AGE
#define SPIFFS_GC_HEUR_W_ERASE_AGE (50)
#endif
// Object name maximum length. Note that this length include the
// zero-termination character, meaning maximum string of characters
// can at most be SPIFFS_OBJ_NAME_LEN - 1.
#ifndef SPIFFS_OBJ_NAME_LEN
#define SPIFFS_OBJ_NAME_LEN (32)
#endif
// Maximum length of the metadata associated with an object.
// Setting to non-zero value enables metadata-related API but also
// changes the on-disk format, so the change is not backward-compatible.
//
// Do note: the meta length must never exceed
// logical_page_size - (SPIFFS_OBJ_NAME_LEN + 64)
//
// This is derived from following:
// logical_page_size - (SPIFFS_OBJ_NAME_LEN + sizeof(spiffs_page_header) +
// spiffs_object_ix_header fields + at least some LUT entries)
#ifndef SPIFFS_OBJ_META_LEN
#define SPIFFS_OBJ_META_LEN (0)
#endif
// Size of buffer allocated on stack used when copying data.
// Lower value generates more read/writes. No meaning having it bigger
// than logical page size.
#ifndef SPIFFS_COPY_BUFFER_STACK
#define SPIFFS_COPY_BUFFER_STACK (64)
#endif
// Enable this to have an identifiable spiffs filesystem. This will look for
// a magic in all sectors to determine if this is a valid spiffs system or
// not on mount point. If not, SPIFFS_format must be called prior to mounting
// again.
#ifndef SPIFFS_USE_MAGIC
#define SPIFFS_USE_MAGIC (1)
#endif
#if SPIFFS_USE_MAGIC
// Only valid when SPIFFS_USE_MAGIC is enabled. If SPIFFS_USE_MAGIC_LENGTH is
// enabled, the magic will also be dependent on the length of the filesystem.
// For example, a filesystem configured and formatted for 4 megabytes will not
// be accepted for mounting with a configuration defining the filesystem as 2
// megabytes.
#ifndef SPIFFS_USE_MAGIC_LENGTH
#define SPIFFS_USE_MAGIC_LENGTH (0)
#endif
#endif
// SPIFFS_LOCK and SPIFFS_UNLOCK protects spiffs from reentrancy on api level
// These should be defined on a multithreaded system
// define this to enter a mutex if you're running on a multithreaded system
#ifndef SPIFFS_LOCK
#define SPIFFS_LOCK(fs)
#endif
// define this to exit a mutex if you're running on a multithreaded system
#ifndef SPIFFS_UNLOCK
#define SPIFFS_UNLOCK(fs)
#endif
// Enable if only one spiffs instance with constant configuration will exist
// on the target. This will reduce calculations, flash and memory accesses.
// Parts of configuration must be defined below instead of at time of mount.
#ifndef SPIFFS_SINGLETON
#define SPIFFS_SINGLETON 0
#endif
#if SPIFFS_SINGLETON
// Instead of giving parameters in config struct, singleton build must
// give parameters in defines below.
#ifndef SPIFFS_CFG_PHYS_SZ
#define SPIFFS_CFG_PHYS_SZ(ignore) (1024*1024*2)
#endif
#ifndef SPIFFS_CFG_PHYS_ERASE_SZ
#define SPIFFS_CFG_PHYS_ERASE_SZ(ignore) (65536)
#endif
#ifndef SPIFFS_CFG_PHYS_ADDR
#define SPIFFS_CFG_PHYS_ADDR(ignore) (0)
#endif
#ifndef SPIFFS_CFG_LOG_PAGE_SZ
#define SPIFFS_CFG_LOG_PAGE_SZ(ignore) (256)
#endif
#ifndef SPIFFS_CFG_LOG_BLOCK_SZ
#define SPIFFS_CFG_LOG_BLOCK_SZ(ignore) (65536)
#endif
#endif
// Enable this if your target needs aligned data for index tables
#ifndef SPIFFS_ALIGNED_OBJECT_INDEX_TABLES
#define SPIFFS_ALIGNED_OBJECT_INDEX_TABLES 1
#endif
// Enable this if you want the HAL callbacks to be called with the spiffs struct
#ifndef SPIFFS_HAL_CALLBACK_EXTRA
#define SPIFFS_HAL_CALLBACK_EXTRA 0
#endif
// Enable this if you want to add an integer offset to all file handles
// (spiffs_file). This is useful if running multiple instances of spiffs on
// same target, in order to recognise to what spiffs instance a file handle
// belongs.
// NB: This adds config field fh_ix_offset in the configuration struct when
// mounting, which must be defined.
#ifndef SPIFFS_FILEHDL_OFFSET
#define SPIFFS_FILEHDL_OFFSET 0
#endif
// Enable this to compile a read only version of spiffs.
// This will reduce binary size of spiffs. All code comprising modification
// of the file system will not be compiled. Some config will be ignored.
// HAL functions for erasing and writing to spi-flash may be null. Cache
// can be disabled for even further binary size reduction (and ram savings).
// Functions modifying the fs will return SPIFFS_ERR_RO_NOT_IMPL.
// If the file system cannot be mounted due to aborted erase operation and
// SPIFFS_USE_MAGIC is enabled, SPIFFS_ERR_RO_ABORTED_OPERATION will be
// returned.
// Might be useful for e.g. bootloaders and such.
#ifndef SPIFFS_READ_ONLY
#define SPIFFS_READ_ONLY 0
#endif
// Enable this to add a temporal file cache using the fd buffer.
// The effects of the cache is that SPIFFS_open will find the file faster in
// certain cases. It will make it a lot easier for spiffs to find files
// opened frequently, reducing number of readings from the spi flash for
// finding those files.
// This will grow each fd by 6 bytes. If your files are opened in patterns
// with a degree of temporal locality, the system is optimized.
// Examples can be letting spiffs serve web content, where one file is the css.
// The css is accessed for each html file that is opened, meaning it is
// accessed almost every second time a file is opened. Another example could be
// a log file that is often opened, written, and closed.
// The size of the cache is number of given file descriptors, as it piggybacks
// on the fd update mechanism. The cache lives in the closed file descriptors.
// When closed, the fd know the whereabouts of the file. Instead of forgetting
// this, the temporal cache will keep handling updates to that file even if the
// fd is closed. If the file is opened again, the location of the file is found
// directly. If all available descriptors become opened, all cache memory is
// lost.
#ifndef SPIFFS_TEMPORAL_FD_CACHE
#define SPIFFS_TEMPORAL_FD_CACHE 1
#endif
// Temporal file cache hit score. Each time a file is opened, all cached files
// will lose one point. If the opened file is found in cache, that entry will
// gain SPIFFS_TEMPORAL_CACHE_HIT_SCORE points. One can experiment with this
// value for the specific access patterns of the application. However, it must
// be between 1 (no gain for hitting a cached entry often) and 255.
#ifndef SPIFFS_TEMPORAL_CACHE_HIT_SCORE
#define SPIFFS_TEMPORAL_CACHE_HIT_SCORE 4
#endif
// Enable to be able to map object indices to memory.
// This allows for faster and more deterministic reading if cases of reading
// large files and when changing file offset by seeking around a lot.
// When mapping a file's index, the file system will be scanned for index pages
// and the info will be put in memory provided by user. When reading, the
// memory map can be looked up instead of searching for index pages on the
// medium. This way, user can trade memory against performance.
// Whole, parts of, or future parts not being written yet can be mapped. The
// memory array will be owned by spiffs and updated accordingly during garbage
// collecting or when modifying the indices. The latter is invoked by when the
// file is modified in some way. The index buffer is tied to the file
// descriptor.
#ifndef SPIFFS_IX_MAP
#define SPIFFS_IX_MAP 1
#endif
// By default SPIFFS in some cases relies on the property of NOR flash that bits
// cannot be set from 0 to 1 by writing and that controllers will ignore such
// bit changes. This results in fewer reads as SPIFFS can in some cases perform
// blind writes, with all bits set to 1 and only those it needs reset set to 0.
// Most of the chips and controllers allow this behavior, so the default is to
// use this technique. If your controller is one of the rare ones that don't,
// turn this option on and SPIFFS will perform a read-modify-write instead.
#ifndef SPIFFS_NO_BLIND_WRITES
#define SPIFFS_NO_BLIND_WRITES 0
#endif
// Set SPIFFS_TEST_VISUALISATION to non-zero to enable SPIFFS_vis function
// in the api. This function will visualize all filesystem using given printf
// function.
#ifndef SPIFFS_TEST_VISUALISATION
#define SPIFFS_TEST_VISUALISATION 1
#endif
#if SPIFFS_TEST_VISUALISATION
#ifndef spiffs_printf
#define spiffs_printf(...) printf(__VA_ARGS__)
#endif
// spiffs_printf argument for a free page
#ifndef SPIFFS_TEST_VIS_FREE_STR
#define SPIFFS_TEST_VIS_FREE_STR "_"
#endif
// spiffs_printf argument for a deleted page
#ifndef SPIFFS_TEST_VIS_DELE_STR
#define SPIFFS_TEST_VIS_DELE_STR "/"
#endif
// spiffs_printf argument for an index page for given object id
#ifndef SPIFFS_TEST_VIS_INDX_STR
#define SPIFFS_TEST_VIS_INDX_STR(id) "i"
#endif
// spiffs_printf argument for a data page for given object id
#ifndef SPIFFS_TEST_VIS_DATA_STR
#define SPIFFS_TEST_VIS_DATA_STR(id) "d"
#endif
#endif
// Types depending on configuration such as the amount of flash bytes
// given to spiffs file system in total (spiffs_file_system_size),
// the logical block size (log_block_size), and the logical page size
// (log_page_size)
// Block index type. Make sure the size of this type can hold
// the highest number of all blocks - i.e. spiffs_file_system_size / log_block_size
typedef u16_t spiffs_block_ix;
// Page index type. Make sure the size of this type can hold
// the highest page number of all pages - i.e. spiffs_file_system_size / log_page_size
typedef u16_t spiffs_page_ix;
// Object id type - most significant bit is reserved for index flag. Make sure the
// size of this type can hold the highest object id on a full system,
// i.e. 2 + (spiffs_file_system_size / (2*log_page_size))*2
typedef u16_t spiffs_obj_id;
// Object span index type. Make sure the size of this type can
// hold the largest possible span index on the system -
// i.e. (spiffs_file_system_size / log_page_size) - 1
typedef u16_t spiffs_span_ix;
#endif /* SPIFFS_CONFIG_H_ */

606
src/spiffs/spiffs_gc.c
Unescape View File

@ -0,0 +1,606 @@
#include "spiffs.h"
#include "spiffs_nucleus.h"
#if !SPIFFS_READ_ONLY
// Erases a logical block and updates the erase counter.
// If cache is enabled, all pages that might be cached in this block
// is dropped.
static s32_t spiffs_gc_erase_block(
spiffs *fs,
spiffs_block_ix bix) {
s32_t res;
SPIFFS_GC_DBG("gc: erase block "_SPIPRIbl"\n", bix);
res = spiffs_erase_block(fs, bix);
SPIFFS_CHECK_RES(res);
#if SPIFFS_CACHE
{
u32_t i;
for (i = 0; i < SPIFFS_PAGES_PER_BLOCK(fs); i++) {
spiffs_cache_drop_page(fs, SPIFFS_PAGE_FOR_BLOCK(fs, bix) + i);
}
}
#endif
return res;
}
// Searches for blocks where all entries are deleted - if one is found,
// the block is erased. Compared to the non-quick gc, the quick one ensures
// that no updates are needed on existing objects on pages that are erased.
s32_t spiffs_gc_quick(
spiffs *fs, u16_t max_free_pages) {
s32_t res = SPIFFS_OK;
u32_t blocks = fs->block_count;
spiffs_block_ix cur_block = 0;
u32_t cur_block_addr = 0;
int cur_entry = 0;
spiffs_obj_id *obj_lu_buf = (spiffs_obj_id *)fs->lu_work;
SPIFFS_GC_DBG("gc_quick: running\n");
#if SPIFFS_GC_STATS
fs->stats_gc_runs++;
#endif
int entries_per_page = (SPIFFS_CFG_LOG_PAGE_SZ(fs) / sizeof(spiffs_obj_id));
// find fully deleted blocks
// check each block
while (res == SPIFFS_OK && blocks--) {
u16_t deleted_pages_in_block = 0;
u16_t free_pages_in_block = 0;
int obj_lookup_page = 0;
// check each object lookup page
while (res == SPIFFS_OK && obj_lookup_page < (int)SPIFFS_OBJ_LOOKUP_PAGES(fs)) {
int entry_offset = obj_lookup_page * entries_per_page;
res = _spiffs_rd(fs, SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_READ,
0, cur_block_addr + SPIFFS_PAGE_TO_PADDR(fs, obj_lookup_page), SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->lu_work);
// check each entry
while (res == SPIFFS_OK &&
cur_entry - entry_offset < entries_per_page &&
cur_entry < (int)(SPIFFS_PAGES_PER_BLOCK(fs)-SPIFFS_OBJ_LOOKUP_PAGES(fs))) {
spiffs_obj_id obj_id = obj_lu_buf[cur_entry-entry_offset];
if (obj_id == SPIFFS_OBJ_ID_DELETED) {
deleted_pages_in_block++;
} else if (obj_id == SPIFFS_OBJ_ID_FREE) {
// kill scan, go for next block
free_pages_in_block++;
if (free_pages_in_block > max_free_pages) {
obj_lookup_page = SPIFFS_OBJ_LOOKUP_PAGES(fs);
res = 1; // kill object lu loop
break;
}
} else {
// kill scan, go for next block
obj_lookup_page = SPIFFS_OBJ_LOOKUP_PAGES(fs);
res = 1; // kill object lu loop
break;
}
cur_entry++;
} // per entry
obj_lookup_page++;
} // per object lookup page
if (res == 1) res = SPIFFS_OK;
if (res == SPIFFS_OK &&
deleted_pages_in_block + free_pages_in_block == SPIFFS_PAGES_PER_BLOCK(fs)-SPIFFS_OBJ_LOOKUP_PAGES(fs) &&
free_pages_in_block <= max_free_pages) {
// found a fully deleted block
fs->stats_p_deleted -= deleted_pages_in_block;
res = spiffs_gc_erase_block(fs, cur_block);
return res;
}
cur_entry = 0;
cur_block++;
cur_block_addr += SPIFFS_CFG_LOG_BLOCK_SZ(fs);
} // per block
if (res == SPIFFS_OK) {
res = SPIFFS_ERR_NO_DELETED_BLOCKS;
}
return res;
}
// Checks if garbage collecting is necessary. If so a candidate block is found,
// cleansed and erased
s32_t spiffs_gc_check(
spiffs *fs,
u32_t len) {
s32_t res;
s32_t free_pages =
(SPIFFS_PAGES_PER_BLOCK(fs) - SPIFFS_OBJ_LOOKUP_PAGES(fs)) * (fs->block_count-2)
- fs->stats_p_allocated - fs->stats_p_deleted;
int tries = 0;
if (fs->free_blocks > 3 &&
(s32_t)len < free_pages * (s32_t)SPIFFS_DATA_PAGE_SIZE(fs)) {
return SPIFFS_OK;
}
u32_t needed_pages = (len + SPIFFS_DATA_PAGE_SIZE(fs) - 1) / SPIFFS_DATA_PAGE_SIZE(fs);
// if (fs->free_blocks <= 2 && (s32_t)needed_pages > free_pages) {
// SPIFFS_GC_DBG("gc: full freeblk:"_SPIPRIi" needed:"_SPIPRIi" free:"_SPIPRIi" dele:"_SPIPRIi"\n", fs->free_blocks, needed_pages, free_pages, fs->stats_p_deleted);
// return SPIFFS_ERR_FULL;
// }
if ((s32_t)needed_pages > (s32_t)(free_pages + fs->stats_p_deleted)) {
SPIFFS_GC_DBG("gc_check: full freeblk:"_SPIPRIi" needed:"_SPIPRIi" free:"_SPIPRIi" dele:"_SPIPRIi"\n", fs->free_blocks, needed_pages, free_pages, fs->stats_p_deleted);
return SPIFFS_ERR_FULL;
}
do {
SPIFFS_GC_DBG("\ngc_check #"_SPIPRIi": run gc free_blocks:"_SPIPRIi" pfree:"_SPIPRIi" pallo:"_SPIPRIi" pdele:"_SPIPRIi" ["_SPIPRIi"] len:"_SPIPRIi" of "_SPIPRIi"\n",
tries,
fs->free_blocks, free_pages, fs->stats_p_allocated, fs->stats_p_deleted, (free_pages+fs->stats_p_allocated+fs->stats_p_deleted),
len, (u32_t)(free_pages*SPIFFS_DATA_PAGE_SIZE(fs)));
spiffs_block_ix *cands;
int count;
spiffs_block_ix cand;
s32_t prev_free_pages = free_pages;
// if the fs is crammed, ignore block age when selecting candidate - kind of a bad state
res = spiffs_gc_find_candidate(fs, &cands, &count, free_pages <= 0);
SPIFFS_CHECK_RES(res);
if (count == 0) {
SPIFFS_GC_DBG("gc_check: no candidates, return\n");
return (s32_t)needed_pages < free_pages ? SPIFFS_OK : SPIFFS_ERR_FULL;
}
#if SPIFFS_GC_STATS
fs->stats_gc_runs++;
#endif
cand = cands[0];
fs->cleaning = 1;
//SPIFFS_GC_DBG("gcing: cleaning block "_SPIPRIi"\n", cand);
res = spiffs_gc_clean(fs, cand);
fs->cleaning = 0;
if (res < 0) {
SPIFFS_GC_DBG("gc_check: cleaning block "_SPIPRIi", result "_SPIPRIi"\n", cand, res);
} else {
SPIFFS_GC_DBG("gc_check: cleaning block "_SPIPRIi", result "_SPIPRIi"\n", cand, res);
}
SPIFFS_CHECK_RES(res);
res = spiffs_gc_erase_page_stats(fs, cand);
SPIFFS_CHECK_RES(res);
res = spiffs_gc_erase_block(fs, cand);
SPIFFS_CHECK_RES(res);
free_pages =
(SPIFFS_PAGES_PER_BLOCK(fs) - SPIFFS_OBJ_LOOKUP_PAGES(fs)) * (fs->block_count - 2)
- fs->stats_p_allocated - fs->stats_p_deleted;
if (prev_free_pages <= 0 && prev_free_pages == free_pages) {
// abort early to reduce wear, at least tried once
SPIFFS_GC_DBG("gc_check: early abort, no result on gc when fs crammed\n");
break;
}
} while (++tries < SPIFFS_GC_MAX_RUNS && (fs->free_blocks <= 2 ||
(s32_t)len > free_pages*(s32_t)SPIFFS_DATA_PAGE_SIZE(fs)));
free_pages =
(SPIFFS_PAGES_PER_BLOCK(fs) - SPIFFS_OBJ_LOOKUP_PAGES(fs)) * (fs->block_count - 2)
- fs->stats_p_allocated - fs->stats_p_deleted;
if ((s32_t)len > free_pages*(s32_t)SPIFFS_DATA_PAGE_SIZE(fs)) {
res = SPIFFS_ERR_FULL;
}
SPIFFS_GC_DBG("gc_check: finished, "_SPIPRIi" dirty, blocks "_SPIPRIi" free, "_SPIPRIi" pages free, "_SPIPRIi" tries, res "_SPIPRIi"\n",
fs->stats_p_allocated + fs->stats_p_deleted,
fs->free_blocks, free_pages, tries, res);
return res;
}
// Updates page statistics for a block that is about to be erased
s32_t spiffs_gc_erase_page_stats(
spiffs *fs,
spiffs_block_ix bix) {
s32_t res = SPIFFS_OK;
int obj_lookup_page = 0;
int entries_per_page = (SPIFFS_CFG_LOG_PAGE_SZ(fs) / sizeof(spiffs_obj_id));
spiffs_obj_id *obj_lu_buf = (spiffs_obj_id *)fs->lu_work;
int cur_entry = 0;
u32_t dele = 0;
u32_t allo = 0;
// check each object lookup page
while (res == SPIFFS_OK && obj_lookup_page < (int)SPIFFS_OBJ_LOOKUP_PAGES(fs)) {
int entry_offset = obj_lookup_page * entries_per_page;
res = _spiffs_rd(fs, SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_READ,
0, bix * SPIFFS_CFG_LOG_BLOCK_SZ(fs) + SPIFFS_PAGE_TO_PADDR(fs, obj_lookup_page), SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->lu_work);
// check each entry
while (res == SPIFFS_OK &&
cur_entry - entry_offset < entries_per_page && cur_entry < (int)(SPIFFS_PAGES_PER_BLOCK(fs)-SPIFFS_OBJ_LOOKUP_PAGES(fs))) {
spiffs_obj_id obj_id = obj_lu_buf[cur_entry-entry_offset];
if (obj_id == SPIFFS_OBJ_ID_FREE) {
} else if (obj_id == SPIFFS_OBJ_ID_DELETED) {
dele++;
} else {
allo++;
}
cur_entry++;
} // per entry
obj_lookup_page++;
} // per object lookup page
SPIFFS_GC_DBG("gc_check: wipe pallo:"_SPIPRIi" pdele:"_SPIPRIi"\n", allo, dele);
fs->stats_p_allocated -= allo;
fs->stats_p_deleted -= dele;
return res;
}
// Finds block candidates to erase
s32_t spiffs_gc_find_candidate(
spiffs *fs,
spiffs_block_ix **block_candidates,
int *candidate_count,
char fs_crammed) {
s32_t res = SPIFFS_OK;
u32_t blocks = fs->block_count;
spiffs_block_ix cur_block = 0;
u32_t cur_block_addr = 0;
spiffs_obj_id *obj_lu_buf = (spiffs_obj_id *)fs->lu_work;
int cur_entry = 0;
// using fs->work area as sorted candidate memory, (spiffs_block_ix)cand_bix/(s32_t)score
int max_candidates = MIN(fs->block_count, (SPIFFS_CFG_LOG_PAGE_SZ(fs)-8)/(sizeof(spiffs_block_ix) + sizeof(s32_t)));
*candidate_count = 0;
memset(fs->work, 0xff, SPIFFS_CFG_LOG_PAGE_SZ(fs));
// divide up work area into block indices and scores
spiffs_block_ix *cand_blocks = (spiffs_block_ix *)fs->work;
s32_t *cand_scores = (s32_t *)(fs->work + max_candidates * sizeof(spiffs_block_ix));
// align cand_scores on s32_t boundary
cand_scores = (s32_t*)(((intptr_t)cand_scores + sizeof(intptr_t) - 1) & ~(sizeof(intptr_t) - 1));
*block_candidates = cand_blocks;
int entries_per_page = (SPIFFS_CFG_LOG_PAGE_SZ(fs) / sizeof(spiffs_obj_id));
// check each block
while (res == SPIFFS_OK && blocks--) {
u16_t deleted_pages_in_block = 0;
u16_t used_pages_in_block = 0;
int obj_lookup_page = 0;
// check each object lookup page
while (res == SPIFFS_OK && obj_lookup_page < (int)SPIFFS_OBJ_LOOKUP_PAGES(fs)) {
int entry_offset = obj_lookup_page * entries_per_page;
res = _spiffs_rd(fs, SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_READ,
0, cur_block_addr + SPIFFS_PAGE_TO_PADDR(fs, obj_lookup_page), SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->lu_work);
// check each entry
while (res == SPIFFS_OK &&
cur_entry - entry_offset < entries_per_page &&
cur_entry < (int)(SPIFFS_PAGES_PER_BLOCK(fs)-SPIFFS_OBJ_LOOKUP_PAGES(fs))) {
spiffs_obj_id obj_id = obj_lu_buf[cur_entry-entry_offset];
if (obj_id == SPIFFS_OBJ_ID_FREE) {
// when a free entry is encountered, scan logic ensures that all following entries are free also
res = 1; // kill object lu loop
break;
} else if (obj_id == SPIFFS_OBJ_ID_DELETED) {
deleted_pages_in_block++;
} else {
used_pages_in_block++;
}
cur_entry++;
} // per entry
obj_lookup_page++;
} // per object lookup page
if (res == 1) res = SPIFFS_OK;
// calculate score and insert into candidate table
// stoneage sort, but probably not so many blocks
if (res == SPIFFS_OK /*&& deleted_pages_in_block > 0*/) {
// read erase count
spiffs_obj_id erase_count;
res = _spiffs_rd(fs, SPIFFS_OP_C_READ | SPIFFS_OP_T_OBJ_LU2, 0,
SPIFFS_ERASE_COUNT_PADDR(fs, cur_block),
sizeof(spiffs_obj_id), (u8_t *)&erase_count);
SPIFFS_CHECK_RES(res);
spiffs_obj_id erase_age;
if (fs->max_erase_count > erase_count) {
erase_age = fs->max_erase_count - erase_count;
} else {
erase_age = SPIFFS_OBJ_ID_FREE - (erase_count - fs->max_erase_count);
}
s32_t score =
deleted_pages_in_block * SPIFFS_GC_HEUR_W_DELET +
used_pages_in_block * SPIFFS_GC_HEUR_W_USED +
erase_age * (fs_crammed ? 0 : SPIFFS_GC_HEUR_W_ERASE_AGE);
int cand_ix = 0;
SPIFFS_GC_DBG("gc_check: bix:"_SPIPRIbl" del:"_SPIPRIi" use:"_SPIPRIi" score:"_SPIPRIi"\n", cur_block, deleted_pages_in_block, used_pages_in_block, score);
while (cand_ix < max_candidates) {
if (cand_blocks[cand_ix] == (spiffs_block_ix)-1) {
cand_blocks[cand_ix] = cur_block;
cand_scores[cand_ix] = score;
break;
} else if (cand_scores[cand_ix] < score) {
int reorder_cand_ix = max_candidates - 2;
while (reorder_cand_ix >= cand_ix) {
cand_blocks[reorder_cand_ix + 1] = cand_blocks[reorder_cand_ix];
cand_scores[reorder_cand_ix + 1] = cand_scores[reorder_cand_ix];
reorder_cand_ix--;
}
cand_blocks[cand_ix] = cur_block;
cand_scores[cand_ix] = score;
break;
}
cand_ix++;
}
(*candidate_count)++;
}
cur_entry = 0;
cur_block++;
cur_block_addr += SPIFFS_CFG_LOG_BLOCK_SZ(fs);
} // per block
return res;
}
typedef enum {
FIND_OBJ_DATA,
MOVE_OBJ_DATA,
MOVE_OBJ_IX,
FINISHED
} spiffs_gc_clean_state;
typedef struct {
spiffs_gc_clean_state state;
spiffs_obj_id cur_obj_id;
spiffs_span_ix cur_objix_spix;
spiffs_page_ix cur_objix_pix;
spiffs_page_ix cur_data_pix;
int stored_scan_entry_index;
u8_t obj_id_found;
} spiffs_gc;
// Empties given block by moving all data into free pages of another block
// Strategy:
// loop:
// scan object lookup for object data pages
// for first found id, check spix and load corresponding object index page to memory
// push object scan lookup entry index
// rescan object lookup, find data pages with same id and referenced by same object index
// move data page, update object index in memory
// when reached end of lookup, store updated object index
// pop object scan lookup entry index
// repeat loop until end of object lookup
// scan object lookup again for remaining object index pages, move to new page in other block
//
s32_t spiffs_gc_clean(spiffs *fs, spiffs_block_ix bix) {
s32_t res = SPIFFS_OK;
const int entries_per_page = (SPIFFS_CFG_LOG_PAGE_SZ(fs) / sizeof(spiffs_obj_id));
// this is the global localizer being pushed and popped
int cur_entry = 0;
spiffs_obj_id *obj_lu_buf = (spiffs_obj_id *)fs->lu_work;
spiffs_gc gc; // our stack frame/state
spiffs_page_ix cur_pix = 0;
spiffs_page_object_ix_header *objix_hdr = (spiffs_page_object_ix_header *)fs->work;
spiffs_page_object_ix *objix = (spiffs_page_object_ix *)fs->work;
SPIFFS_GC_DBG("gc_clean: cleaning block "_SPIPRIbl"\n", bix);
memset(&gc, 0, sizeof(spiffs_gc));
gc.state = FIND_OBJ_DATA;
if (fs->free_cursor_block_ix == bix) {
// move free cursor to next block, cannot use free pages from the block we want to clean
fs->free_cursor_block_ix = (bix+1)%fs->block_count;
fs->free_cursor_obj_lu_entry = 0;
SPIFFS_GC_DBG("gc_clean: move free cursor to block "_SPIPRIbl"\n", fs->free_cursor_block_ix);
}
while (res == SPIFFS_OK && gc.state != FINISHED) {
SPIFFS_GC_DBG("gc_clean: state = "_SPIPRIi" entry:"_SPIPRIi"\n", gc.state, cur_entry);
gc.obj_id_found = 0; // reset (to no found data page)
// scan through lookup pages
int obj_lookup_page = cur_entry / entries_per_page;
u8_t scan = 1;
// check each object lookup page
while (scan && res == SPIFFS_OK && obj_lookup_page < (int)SPIFFS_OBJ_LOOKUP_PAGES(fs)) {
int entry_offset = obj_lookup_page * entries_per_page;
res = _spiffs_rd(fs, SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_READ,
0, bix * SPIFFS_CFG_LOG_BLOCK_SZ(fs) + SPIFFS_PAGE_TO_PADDR(fs, obj_lookup_page),
SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->lu_work);
// check each object lookup entry
while (scan && res == SPIFFS_OK &&
cur_entry - entry_offset < entries_per_page && cur_entry < (int)(SPIFFS_PAGES_PER_BLOCK(fs)-SPIFFS_OBJ_LOOKUP_PAGES(fs))) {
spiffs_obj_id obj_id = obj_lu_buf[cur_entry-entry_offset];
cur_pix = SPIFFS_OBJ_LOOKUP_ENTRY_TO_PIX(fs, bix, cur_entry);
// act upon object id depending on gc state
switch (gc.state) {
case FIND_OBJ_DATA:
// find a data page
if (obj_id != SPIFFS_OBJ_ID_DELETED && obj_id != SPIFFS_OBJ_ID_FREE &&
((obj_id & SPIFFS_OBJ_ID_IX_FLAG) == 0)) {
// found a data page, stop scanning and handle in switch case below
SPIFFS_GC_DBG("gc_clean: FIND_DATA state:"_SPIPRIi" - found obj id "_SPIPRIid"\n", gc.state, obj_id);
gc.obj_id_found = 1;
gc.cur_obj_id = obj_id;
gc.cur_data_pix = cur_pix;
scan = 0;
}
break;
case MOVE_OBJ_DATA:
// evacuate found data pages for corresponding object index we have in memory,
// update memory representation
if (obj_id == gc.cur_obj_id) {
spiffs_page_header p_hdr;
res = _spiffs_rd(fs, SPIFFS_OP_T_OBJ_LU2 | SPIFFS_OP_C_READ,
0, SPIFFS_PAGE_TO_PADDR(fs, cur_pix), sizeof(spiffs_page_header), (u8_t*)&p_hdr);
SPIFFS_CHECK_RES(res);
SPIFFS_GC_DBG("gc_clean: MOVE_DATA found data page "_SPIPRIid":"_SPIPRIsp" @ "_SPIPRIpg"\n", gc.cur_obj_id, p_hdr.span_ix, cur_pix);
if (SPIFFS_OBJ_IX_ENTRY_SPAN_IX(fs, p_hdr.span_ix) != gc.cur_objix_spix) {
SPIFFS_GC_DBG("gc_clean: MOVE_DATA no objix spix match, take in another run\n");
} else {
spiffs_page_ix new_data_pix;
if (p_hdr.flags & SPIFFS_PH_FLAG_DELET) {
// move page
res = spiffs_page_move(fs, 0, 0, obj_id, &p_hdr, cur_pix, &new_data_pix);
SPIFFS_GC_DBG("gc_clean: MOVE_DATA move objix "_SPIPRIid":"_SPIPRIsp" page "_SPIPRIpg" to "_SPIPRIpg"\n", gc.cur_obj_id, p_hdr.span_ix, cur_pix, new_data_pix);
SPIFFS_CHECK_RES(res);
// move wipes obj_lu, reload it
res = _spiffs_rd(fs, SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_READ,
0, bix * SPIFFS_CFG_LOG_BLOCK_SZ(fs) + SPIFFS_PAGE_TO_PADDR(fs, obj_lookup_page),
SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->lu_work);
SPIFFS_CHECK_RES(res);
} else {
// page is deleted but not deleted in lookup, scrap it -
// might seem unnecessary as we will erase this block, but
// we might get aborted
SPIFFS_GC_DBG("gc_clean: MOVE_DATA wipe objix "_SPIPRIid":"_SPIPRIsp" page "_SPIPRIpg"\n", obj_id, p_hdr.span_ix, cur_pix);
res = spiffs_page_delete(fs, cur_pix);
SPIFFS_CHECK_RES(res);
new_data_pix = SPIFFS_OBJ_ID_FREE;
}
// update memory representation of object index page with new data page
if (gc.cur_objix_spix == 0) {
// update object index header page
((spiffs_page_ix*)((u8_t *)objix_hdr + sizeof(spiffs_page_object_ix_header)))[p_hdr.span_ix] = new_data_pix;
SPIFFS_GC_DBG("gc_clean: MOVE_DATA wrote page "_SPIPRIpg" to objix_hdr entry "_SPIPRIsp" in mem\n", new_data_pix, (spiffs_span_ix)SPIFFS_OBJ_IX_ENTRY(fs, p_hdr.span_ix));
} else {
// update object index page
((spiffs_page_ix*)((u8_t *)objix + sizeof(spiffs_page_object_ix)))[SPIFFS_OBJ_IX_ENTRY(fs, p_hdr.span_ix)] = new_data_pix;
SPIFFS_GC_DBG("gc_clean: MOVE_DATA wrote page "_SPIPRIpg" to objix entry "_SPIPRIsp" in mem\n", new_data_pix, (spiffs_span_ix)SPIFFS_OBJ_IX_ENTRY(fs, p_hdr.span_ix));
}
}
}
break;
case MOVE_OBJ_IX:
// find and evacuate object index pages
if (obj_id != SPIFFS_OBJ_ID_DELETED && obj_id != SPIFFS_OBJ_ID_FREE &&
(obj_id & SPIFFS_OBJ_ID_IX_FLAG)) {
// found an index object id
spiffs_page_header p_hdr;
spiffs_page_ix new_pix;
// load header
res = _spiffs_rd(fs, SPIFFS_OP_T_OBJ_LU2 | SPIFFS_OP_C_READ,
0, SPIFFS_PAGE_TO_PADDR(fs, cur_pix), sizeof(spiffs_page_header), (u8_t*)&p_hdr);
SPIFFS_CHECK_RES(res);
if (p_hdr.flags & SPIFFS_PH_FLAG_DELET) {
// move page
res = spiffs_page_move(fs, 0, 0, obj_id, &p_hdr, cur_pix, &new_pix);
SPIFFS_GC_DBG("gc_clean: MOVE_OBJIX move objix "_SPIPRIid":"_SPIPRIsp" page "_SPIPRIpg" to "_SPIPRIpg"\n", obj_id, p_hdr.span_ix, cur_pix, new_pix);
SPIFFS_CHECK_RES(res);
spiffs_cb_object_event(fs, (spiffs_page_object_ix *)&p_hdr,
SPIFFS_EV_IX_MOV, obj_id, p_hdr.span_ix, new_pix, 0);
// move wipes obj_lu, reload it
res = _spiffs_rd(fs, SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_READ,
0, bix * SPIFFS_CFG_LOG_BLOCK_SZ(fs) + SPIFFS_PAGE_TO_PADDR(fs, obj_lookup_page),
SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->lu_work);
SPIFFS_CHECK_RES(res);
} else {
// page is deleted but not deleted in lookup, scrap it -
// might seem unnecessary as we will erase this block, but
// we might get aborted
SPIFFS_GC_DBG("gc_clean: MOVE_OBJIX wipe objix "_SPIPRIid":"_SPIPRIsp" page "_SPIPRIpg"\n", obj_id, p_hdr.span_ix, cur_pix);
res = spiffs_page_delete(fs, cur_pix);
if (res == SPIFFS_OK) {
spiffs_cb_object_event(fs, (spiffs_page_object_ix *)0,
SPIFFS_EV_IX_DEL, obj_id, p_hdr.span_ix, cur_pix, 0);
}
}
SPIFFS_CHECK_RES(res);
}
break;
default:
scan = 0;
break;
} // switch gc state
cur_entry++;
} // per entry
obj_lookup_page++; // no need to check scan variable here, obj_lookup_page is set in start of loop
} // per object lookup page
if (res != SPIFFS_OK) break;
// state finalization and switch
switch (gc.state) {
case FIND_OBJ_DATA:
if (gc.obj_id_found) {
// handle found data page -
// find out corresponding obj ix page and load it to memory
spiffs_page_header p_hdr;
spiffs_page_ix objix_pix;
gc.stored_scan_entry_index = cur_entry; // push cursor
cur_entry = 0; // restart scan from start
gc.state = MOVE_OBJ_DATA;
res = _spiffs_rd(fs, SPIFFS_OP_T_OBJ_LU2 | SPIFFS_OP_C_READ,
0, SPIFFS_PAGE_TO_PADDR(fs, cur_pix), sizeof(spiffs_page_header), (u8_t*)&p_hdr);
SPIFFS_CHECK_RES(res);
gc.cur_objix_spix = SPIFFS_OBJ_IX_ENTRY_SPAN_IX(fs, p_hdr.span_ix);
SPIFFS_GC_DBG("gc_clean: FIND_DATA find objix span_ix:"_SPIPRIsp"\n", gc.cur_objix_spix);
res = spiffs_obj_lu_find_id_and_span(fs, gc.cur_obj_id | SPIFFS_OBJ_ID_IX_FLAG, gc.cur_objix_spix, 0, &objix_pix);
if (res == SPIFFS_ERR_NOT_FOUND) {
// on borked systems we might get an ERR_NOT_FOUND here -
// this is handled by simply deleting the page as it is not referenced
// from anywhere
SPIFFS_GC_DBG("gc_clean: FIND_OBJ_DATA objix not found! Wipe page "_SPIPRIpg"\n", gc.cur_data_pix);
res = spiffs_page_delete(fs, gc.cur_data_pix);
SPIFFS_CHECK_RES(res);
// then we restore states and continue scanning for data pages
cur_entry = gc.stored_scan_entry_index; // pop cursor
gc.state = FIND_OBJ_DATA;
break; // done
}
SPIFFS_CHECK_RES(res);
SPIFFS_GC_DBG("gc_clean: FIND_DATA found object index at page "_SPIPRIpg"\n", objix_pix);
res = _spiffs_rd(fs, SPIFFS_OP_T_OBJ_LU2 | SPIFFS_OP_C_READ,
0, SPIFFS_PAGE_TO_PADDR(fs, objix_pix), SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->work);
SPIFFS_CHECK_RES(res);
// cannot allow a gc if the presumed index in fact is no index, a
// check must run or lot of data may be lost
SPIFFS_VALIDATE_OBJIX(objix->p_hdr, gc.cur_obj_id | SPIFFS_OBJ_ID_IX_FLAG, gc.cur_objix_spix);
gc.cur_objix_pix = objix_pix;
} else {
// no more data pages found, passed thru all block, start evacuating object indices
gc.state = MOVE_OBJ_IX;
cur_entry = 0; // restart entry scan index
}
break;
case MOVE_OBJ_DATA: {
// store modified objix (hdr) page residing in memory now that all
// data pages belonging to this object index and residing in the block
// we want to evacuate
spiffs_page_ix new_objix_pix;
gc.state = FIND_OBJ_DATA;
cur_entry = gc.stored_scan_entry_index; // pop cursor
if (gc.cur_objix_spix == 0) {
// store object index header page
res = spiffs_object_update_index_hdr(fs, 0, gc.cur_obj_id | SPIFFS_OBJ_ID_IX_FLAG, gc.cur_objix_pix, fs->work, 0, 0, 0, &new_objix_pix);
SPIFFS_GC_DBG("gc_clean: MOVE_DATA store modified objix_hdr page, "_SPIPRIpg":"_SPIPRIsp"\n", new_objix_pix, 0);
SPIFFS_CHECK_RES(res);
} else {
// store object index page
res = spiffs_page_move(fs, 0, fs->work, gc.cur_obj_id | SPIFFS_OBJ_ID_IX_FLAG, 0, gc.cur_objix_pix, &new_objix_pix);
SPIFFS_GC_DBG("gc_clean: MOVE_DATA store modified objix page, "_SPIPRIpg":"_SPIPRIsp"\n", new_objix_pix, objix->p_hdr.span_ix);
SPIFFS_CHECK_RES(res);
spiffs_cb_object_event(fs, (spiffs_page_object_ix *)fs->work,
SPIFFS_EV_IX_UPD, gc.cur_obj_id, objix->p_hdr.span_ix, new_objix_pix, 0);
}
}
break;
case MOVE_OBJ_IX:
// scanned thru all block, no more object indices found - our work here is done
gc.state = FINISHED;
break;
default:
cur_entry = 0;
break;
} // switch gc.state
SPIFFS_GC_DBG("gc_clean: state-> "_SPIPRIi"\n", gc.state);
} // while state != FINISHED
return res;
}
#endif // !SPIFFS_READ_ONLY

1451
src/spiffs/spiffs_hydrogen.c
View File

File diff suppressed because it is too large Load Diff

2364
src/spiffs/spiffs_nucleus.c
View File

File diff suppressed because it is too large Load Diff

842
src/spiffs/spiffs_nucleus.h
Unescape View File

@ -0,0 +1,842 @@
/*
* spiffs_nucleus.h
*
* Created on: Jun 15, 2013
* Author: petera
*/
/* SPIFFS layout
*
* spiffs is designed for following spi flash characteristics:
* - only big areas of data (blocks) can be erased
* - erasing resets all bits in a block to ones
* - writing pulls ones to zeroes
* - zeroes cannot be pulled to ones, without erase
* - wear leveling
*
* spiffs is also meant to be run on embedded, memory constraint devices.
*
* Entire area is divided in blocks. Entire area is also divided in pages.
* Each block contains same number of pages. A page cannot be erased, but a
* block can be erased.
*
* Entire area must be block_size * x
* page_size must be block_size / (2^y) where y > 2
*
* ex: area = 1024*1024 bytes, block size = 65536 bytes, page size = 256 bytes
*
* BLOCK 0 PAGE 0 object lookup 1
* PAGE 1 object lookup 2
* ...
* PAGE n-1 object lookup n
* PAGE n object data 1
* PAGE n+1 object data 2
* ...
* PAGE n+m-1 object data m
*
* BLOCK 1 PAGE n+m object lookup 1
* PAGE n+m+1 object lookup 2
* ...
* PAGE 2n+m-1 object lookup n
* PAGE 2n+m object data 1
* PAGE 2n+m object data 2
* ...
* PAGE 2n+2m-1 object data m
* ...
*
* n is number of object lookup pages, which is number of pages needed to index all pages
* in a block by object id
* : block_size / page_size * sizeof(obj_id) / page_size
* m is number data pages, which is number of pages in block minus number of lookup pages
* : block_size / page_size - block_size / page_size * sizeof(obj_id) / page_size
* thus, n+m is total number of pages in a block
* : block_size / page_size
*
* ex: n = 65536/256*2/256 = 2, m = 65536/256 - 2 = 254 => n+m = 65536/256 = 256
*
* Object lookup pages contain object id entries. Each entry represent the corresponding
* data page.
* Assuming a 16 bit object id, an object id being 0xffff represents a free page.
* An object id being 0x0000 represents a deleted page.
*
* ex: page 0 : lookup : 0008 0001 0aaa ffff ffff ffff ffff ffff ..
* page 1 : lookup : ffff ffff ffff ffff ffff ffff ffff ffff ..
* page 2 : data : data for object id 0008
* page 3 : data : data for object id 0001
* page 4 : data : data for object id 0aaa
* ...
*
*
* Object data pages can be either object index pages or object content.
* All object data pages contains a data page header, containing object id and span index.
* The span index denotes the object page ordering amongst data pages with same object id.
* This applies to both object index pages (when index spans more than one page of entries),
* and object data pages.
* An object index page contains page entries pointing to object content page. The entry index
* in a object index page correlates to the span index in the actual object data page.
* The first object index page (span index 0) is called object index header page, and also
* contains object flags (directory/file), size, object name etc.
*
* ex:
* BLOCK 1
* PAGE 256: objectl lookup page 1
* [*123] [ 123] [ 123] [ 123]
* [ 123] [*123] [ 123] [ 123]
* [free] [free] [free] [free] ...
* PAGE 257: objectl lookup page 2
* [free] [free] [free] [free] ...
* PAGE 258: object index page (header)
* obj.id:0123 span.ix:0000 flags:INDEX
* size:1600 name:ex.txt type:file
* [259] [260] [261] [262]
* PAGE 259: object data page
* obj.id:0123 span.ix:0000 flags:DATA
* PAGE 260: object data page
* obj.id:0123 span.ix:0001 flags:DATA
* PAGE 261: object data page
* obj.id:0123 span.ix:0002 flags:DATA
* PAGE 262: object data page
* obj.id:0123 span.ix:0003 flags:DATA
* PAGE 263: object index page
* obj.id:0123 span.ix:0001 flags:INDEX
* [264] [265] [fre] [fre]
* [fre] [fre] [fre] [fre]
* PAGE 264: object data page
* obj.id:0123 span.ix:0004 flags:DATA
* PAGE 265: object data page
* obj.id:0123 span.ix:0005 flags:DATA
*
*/
#ifndef SPIFFS_NUCLEUS_H_
#define SPIFFS_NUCLEUS_H_
#define _SPIFFS_ERR_CHECK_FIRST (SPIFFS_ERR_INTERNAL - 1)
#define SPIFFS_ERR_CHECK_OBJ_ID_MISM (SPIFFS_ERR_INTERNAL - 1)
#define SPIFFS_ERR_CHECK_SPIX_MISM (SPIFFS_ERR_INTERNAL - 2)
#define SPIFFS_ERR_CHECK_FLAGS_BAD (SPIFFS_ERR_INTERNAL - 3)
#define _SPIFFS_ERR_CHECK_LAST (SPIFFS_ERR_INTERNAL - 4)
// visitor result, continue searching
#define SPIFFS_VIS_COUNTINUE (SPIFFS_ERR_INTERNAL - 20)
// visitor result, continue searching after reloading lu buffer
#define SPIFFS_VIS_COUNTINUE_RELOAD (SPIFFS_ERR_INTERNAL - 21)
// visitor result, stop searching
#define SPIFFS_VIS_END (SPIFFS_ERR_INTERNAL - 22)
// updating an object index contents
#define SPIFFS_EV_IX_UPD (0)
// creating a new object index
#define SPIFFS_EV_IX_NEW (1)
// deleting an object index
#define SPIFFS_EV_IX_DEL (2)
// moving an object index without updating contents
#define SPIFFS_EV_IX_MOV (3)
// updating an object index header data only, not the table itself
#define SPIFFS_EV_IX_UPD_HDR (4)
#define SPIFFS_OBJ_ID_IX_FLAG ((spiffs_obj_id)(1<<(8*sizeof(spiffs_obj_id)-1)))
#define SPIFFS_UNDEFINED_LEN (u32_t)(-1)
#define SPIFFS_OBJ_ID_DELETED ((spiffs_obj_id)0)
#define SPIFFS_OBJ_ID_FREE ((spiffs_obj_id)-1)
#if defined(__GNUC__) || defined(__clang__) || defined(__TI_COMPILER_VERSION__)
/* For GCC, clang and TI compilers */
#define SPIFFS_PACKED __attribute__((packed))
#elif defined(__ICCARM__) || defined(__CC_ARM)
/* For IAR ARM and Keil MDK-ARM compilers */
#define SPIFFS_PACKED
#else
/* Unknown compiler */
#define SPIFFS_PACKED
#endif
#if SPIFFS_USE_MAGIC
#if !SPIFFS_USE_MAGIC_LENGTH
#define SPIFFS_MAGIC(fs, bix) \
((spiffs_obj_id)(0x20140529 ^ SPIFFS_CFG_LOG_PAGE_SZ(fs)))
#else // SPIFFS_USE_MAGIC_LENGTH
#define SPIFFS_MAGIC(fs, bix) \
((spiffs_obj_id)(0x20140529 ^ SPIFFS_CFG_LOG_PAGE_SZ(fs) ^ ((fs)->block_count - (bix))))
#endif // SPIFFS_USE_MAGIC_LENGTH
#endif // SPIFFS_USE_MAGIC
#define SPIFFS_CONFIG_MAGIC (0x20090315)
#if SPIFFS_SINGLETON == 0
#define SPIFFS_CFG_LOG_PAGE_SZ(fs) \
((fs)->cfg.log_page_size)
#define SPIFFS_CFG_LOG_BLOCK_SZ(fs) \
((fs)->cfg.log_block_size)
#define SPIFFS_CFG_PHYS_SZ(fs) \
((fs)->cfg.phys_size)
#define SPIFFS_CFG_PHYS_ERASE_SZ(fs) \
((fs)->cfg.phys_erase_block)
#define SPIFFS_CFG_PHYS_ADDR(fs) \
((fs)->cfg.phys_addr)
#endif
// total number of pages
#define SPIFFS_MAX_PAGES(fs) \
( SPIFFS_CFG_PHYS_SZ(fs)/SPIFFS_CFG_LOG_PAGE_SZ(fs) )
// total number of pages per block, including object lookup pages
#define SPIFFS_PAGES_PER_BLOCK(fs) \
( SPIFFS_CFG_LOG_BLOCK_SZ(fs)/SPIFFS_CFG_LOG_PAGE_SZ(fs) )
// number of object lookup pages per block
#define SPIFFS_OBJ_LOOKUP_PAGES(fs) \
(MAX(1, (SPIFFS_PAGES_PER_BLOCK(fs) * sizeof(spiffs_obj_id)) / SPIFFS_CFG_LOG_PAGE_SZ(fs)) )
// checks if page index belongs to object lookup
#define SPIFFS_IS_LOOKUP_PAGE(fs,pix) \
(((pix) % SPIFFS_PAGES_PER_BLOCK(fs)) < SPIFFS_OBJ_LOOKUP_PAGES(fs))
// number of object lookup entries in all object lookup pages
#define SPIFFS_OBJ_LOOKUP_MAX_ENTRIES(fs) \
(SPIFFS_PAGES_PER_BLOCK(fs)-SPIFFS_OBJ_LOOKUP_PAGES(fs))
// converts a block to physical address
#define SPIFFS_BLOCK_TO_PADDR(fs, block) \
( SPIFFS_CFG_PHYS_ADDR(fs) + (block)* SPIFFS_CFG_LOG_BLOCK_SZ(fs) )
// converts a object lookup entry to page index
#define SPIFFS_OBJ_LOOKUP_ENTRY_TO_PIX(fs, block, entry) \
((block)*SPIFFS_PAGES_PER_BLOCK(fs) + (SPIFFS_OBJ_LOOKUP_PAGES(fs) + entry))
// converts a object lookup entry to physical address of corresponding page
#define SPIFFS_OBJ_LOOKUP_ENTRY_TO_PADDR(fs, block, entry) \
(SPIFFS_BLOCK_TO_PADDR(fs, block) + (SPIFFS_OBJ_LOOKUP_PAGES(fs) + entry) * SPIFFS_CFG_LOG_PAGE_SZ(fs) )
// converts a page to physical address
#define SPIFFS_PAGE_TO_PADDR(fs, page) \
( SPIFFS_CFG_PHYS_ADDR(fs) + (page) * SPIFFS_CFG_LOG_PAGE_SZ(fs) )
// converts a physical address to page
#define SPIFFS_PADDR_TO_PAGE(fs, addr) \
( ((addr) - SPIFFS_CFG_PHYS_ADDR(fs)) / SPIFFS_CFG_LOG_PAGE_SZ(fs) )
// gives index in page for a physical address
#define SPIFFS_PADDR_TO_PAGE_OFFSET(fs, addr) \
( ((addr) - SPIFFS_CFG_PHYS_ADDR(fs)) % SPIFFS_CFG_LOG_PAGE_SZ(fs) )
// returns containing block for given page
#define SPIFFS_BLOCK_FOR_PAGE(fs, page) \
( (page) / SPIFFS_PAGES_PER_BLOCK(fs) )
// returns starting page for block
#define SPIFFS_PAGE_FOR_BLOCK(fs, block) \
( (block) * SPIFFS_PAGES_PER_BLOCK(fs) )
// converts page to entry in object lookup page
#define SPIFFS_OBJ_LOOKUP_ENTRY_FOR_PAGE(fs, page) \
( (page) % SPIFFS_PAGES_PER_BLOCK(fs) - SPIFFS_OBJ_LOOKUP_PAGES(fs) )
// returns data size in a data page
#define SPIFFS_DATA_PAGE_SIZE(fs) \
( SPIFFS_CFG_LOG_PAGE_SZ(fs) - sizeof(spiffs_page_header) )
// returns physical address for block's erase count,
// always in the physical last entry of the last object lookup page
#define SPIFFS_ERASE_COUNT_PADDR(fs, bix) \
( SPIFFS_BLOCK_TO_PADDR(fs, bix) + SPIFFS_OBJ_LOOKUP_PAGES(fs) * SPIFFS_CFG_LOG_PAGE_SZ(fs) - sizeof(spiffs_obj_id) )
// returns physical address for block's magic,
// always in the physical second last entry of the last object lookup page
#define SPIFFS_MAGIC_PADDR(fs, bix) \
( SPIFFS_BLOCK_TO_PADDR(fs, bix) + SPIFFS_OBJ_LOOKUP_PAGES(fs) * SPIFFS_CFG_LOG_PAGE_SZ(fs) - sizeof(spiffs_obj_id)*2 )
// checks if there is any room for magic in the object luts
#define SPIFFS_CHECK_MAGIC_POSSIBLE(fs) \
( (SPIFFS_OBJ_LOOKUP_MAX_ENTRIES(fs) % (SPIFFS_CFG_LOG_PAGE_SZ(fs)/sizeof(spiffs_obj_id))) * sizeof(spiffs_obj_id) \
<= (SPIFFS_CFG_LOG_PAGE_SZ(fs)-sizeof(spiffs_obj_id)*2) )
// define helpers object
// entries in an object header page index
#define SPIFFS_OBJ_HDR_IX_LEN(fs) \
((SPIFFS_CFG_LOG_PAGE_SZ(fs) - sizeof(spiffs_page_object_ix_header))/sizeof(spiffs_page_ix))
// entries in an object page index
#define SPIFFS_OBJ_IX_LEN(fs) \
((SPIFFS_CFG_LOG_PAGE_SZ(fs) - sizeof(spiffs_page_object_ix))/sizeof(spiffs_page_ix))
// object index entry for given data span index
#define SPIFFS_OBJ_IX_ENTRY(fs, spix) \
((spix) < SPIFFS_OBJ_HDR_IX_LEN(fs) ? (spix) : (((spix)-SPIFFS_OBJ_HDR_IX_LEN(fs))%SPIFFS_OBJ_IX_LEN(fs)))
// object index span index number for given data span index or entry
#define SPIFFS_OBJ_IX_ENTRY_SPAN_IX(fs, spix) \
((spix) < SPIFFS_OBJ_HDR_IX_LEN(fs) ? 0 : (1+((spix)-SPIFFS_OBJ_HDR_IX_LEN(fs))/SPIFFS_OBJ_IX_LEN(fs)))
// get data span index for object index span index
#define SPIFFS_DATA_SPAN_IX_FOR_OBJ_IX_SPAN_IX(fs, spix) \
( (spix) == 0 ? 0 : (SPIFFS_OBJ_HDR_IX_LEN(fs) + (((spix)-1) * SPIFFS_OBJ_IX_LEN(fs))) )
#if SPIFFS_FILEHDL_OFFSET
#define SPIFFS_FH_OFFS(fs, fh) ((fh) != 0 ? ((fh) + (fs)->cfg.fh_ix_offset) : 0)
#define SPIFFS_FH_UNOFFS(fs, fh) ((fh) != 0 ? ((fh) - (fs)->cfg.fh_ix_offset) : 0)
#else
#define SPIFFS_FH_OFFS(fs, fh) (fh)
#define SPIFFS_FH_UNOFFS(fs, fh) (fh)
#endif
#define SPIFFS_OP_T_OBJ_LU (0<<0)
#define SPIFFS_OP_T_OBJ_LU2 (1<<0)
#define SPIFFS_OP_T_OBJ_IX (2<<0)
#define SPIFFS_OP_T_OBJ_DA (3<<0)
#define SPIFFS_OP_C_DELE (0<<2)
#define SPIFFS_OP_C_UPDT (1<<2)
#define SPIFFS_OP_C_MOVS (2<<2)
#define SPIFFS_OP_C_MOVD (3<<2)
#define SPIFFS_OP_C_FLSH (4<<2)
#define SPIFFS_OP_C_READ (5<<2)
#define SPIFFS_OP_C_WRTHRU (6<<2)
#define SPIFFS_OP_TYPE_MASK (3<<0)
#define SPIFFS_OP_COM_MASK (7<<2)
// if 0, this page is written to, else clean
#define SPIFFS_PH_FLAG_USED (1<<0)
// if 0, writing is finalized, else under modification
#define SPIFFS_PH_FLAG_FINAL (1<<1)
// if 0, this is an index page, else a data page
#define SPIFFS_PH_FLAG_INDEX (1<<2)
// if 0, page is deleted, else valid
#define SPIFFS_PH_FLAG_DELET (1<<7)
// if 0, this index header is being deleted
#define SPIFFS_PH_FLAG_IXDELE (1<<6)
#define SPIFFS_CHECK_MOUNT(fs) \
((fs)->mounted != 0)
#define SPIFFS_CHECK_CFG(fs) \
((fs)->config_magic == SPIFFS_CONFIG_MAGIC)
#define SPIFFS_CHECK_RES(res) \
do { \
if ((res) < SPIFFS_OK) return (res); \
} while (0);
#define SPIFFS_API_CHECK_MOUNT(fs) \
if (!SPIFFS_CHECK_MOUNT((fs))) { \
(fs)->err_code = SPIFFS_ERR_NOT_MOUNTED; \
return SPIFFS_ERR_NOT_MOUNTED; \
}
#define SPIFFS_API_CHECK_CFG(fs) \
if (!SPIFFS_CHECK_CFG((fs))) { \
(fs)->err_code = SPIFFS_ERR_NOT_CONFIGURED; \
return SPIFFS_ERR_NOT_CONFIGURED; \
}
#define SPIFFS_API_CHECK_RES(fs, res) \
if ((res) < SPIFFS_OK) { \
(fs)->err_code = (res); \
return (res); \
}
#define SPIFFS_API_CHECK_RES_UNLOCK(fs, res) \
if ((res) < SPIFFS_OK) { \
(fs)->err_code = (res); \
SPIFFS_UNLOCK(fs); \
return (res); \
}
#define SPIFFS_VALIDATE_OBJIX(ph, objid, spix) \
if (((ph).flags & SPIFFS_PH_FLAG_USED) != 0) return SPIFFS_ERR_IS_FREE; \
if (((ph).flags & SPIFFS_PH_FLAG_DELET) == 0) return SPIFFS_ERR_DELETED; \
if (((ph).flags & SPIFFS_PH_FLAG_FINAL) != 0) return SPIFFS_ERR_NOT_FINALIZED; \
if (((ph).flags & SPIFFS_PH_FLAG_INDEX) != 0) return SPIFFS_ERR_NOT_INDEX; \
if (((objid) & SPIFFS_OBJ_ID_IX_FLAG) == 0) return SPIFFS_ERR_NOT_INDEX; \
if ((ph).span_ix != (spix)) return SPIFFS_ERR_INDEX_SPAN_MISMATCH;
//if ((spix) == 0 && ((ph).flags & SPIFFS_PH_FLAG_IXDELE) == 0) return SPIFFS_ERR_DELETED;
#define SPIFFS_VALIDATE_DATA(ph, objid, spix) \
if (((ph).flags & SPIFFS_PH_FLAG_USED) != 0) return SPIFFS_ERR_IS_FREE; \
if (((ph).flags & SPIFFS_PH_FLAG_DELET) == 0) return SPIFFS_ERR_DELETED; \
if (((ph).flags & SPIFFS_PH_FLAG_FINAL) != 0) return SPIFFS_ERR_NOT_FINALIZED; \
if (((ph).flags & SPIFFS_PH_FLAG_INDEX) == 0) return SPIFFS_ERR_IS_INDEX; \
if ((objid) & SPIFFS_OBJ_ID_IX_FLAG) return SPIFFS_ERR_IS_INDEX; \
if ((ph).span_ix != (spix)) return SPIFFS_ERR_DATA_SPAN_MISMATCH;
// check id, only visit matching objec ids
#define SPIFFS_VIS_CHECK_ID (1<<0)
// report argument object id to visitor - else object lookup id is reported
#define SPIFFS_VIS_CHECK_PH (1<<1)
// stop searching at end of all look up pages
#define SPIFFS_VIS_NO_WRAP (1<<2)
#if SPIFFS_HAL_CALLBACK_EXTRA
#define SPIFFS_HAL_WRITE(_fs, _paddr, _len, _src) \
(_fs)->cfg.hal_write_f((_fs), (_paddr), (_len), (_src))
#define SPIFFS_HAL_READ(_fs, _paddr, _len, _dst) \
(_fs)->cfg.hal_read_f((_fs), (_paddr), (_len), (_dst))
#define SPIFFS_HAL_ERASE(_fs, _paddr, _len) \
(_fs)->cfg.hal_erase_f((_fs), (_paddr), (_len))
#else // SPIFFS_HAL_CALLBACK_EXTRA
#define SPIFFS_HAL_WRITE(_fs, _paddr, _len, _src) \
(_fs)->cfg.hal_write_f((_paddr), (_len), (_src))
#define SPIFFS_HAL_READ(_fs, _paddr, _len, _dst) \
(_fs)->cfg.hal_read_f((_paddr), (_len), (_dst))
#define SPIFFS_HAL_ERASE(_fs, _paddr, _len) \
(_fs)->cfg.hal_erase_f((_paddr), (_len))
#endif // SPIFFS_HAL_CALLBACK_EXTRA
#if SPIFFS_CACHE
#define SPIFFS_CACHE_FLAG_DIRTY (1<<0)
#define SPIFFS_CACHE_FLAG_WRTHRU (1<<1)
#define SPIFFS_CACHE_FLAG_OBJLU (1<<2)
#define SPIFFS_CACHE_FLAG_OBJIX (1<<3)
#define SPIFFS_CACHE_FLAG_DATA (1<<4)
#define SPIFFS_CACHE_FLAG_TYPE_WR (1<<7)
#define SPIFFS_CACHE_PAGE_SIZE(fs) \
(sizeof(spiffs_cache_page) + SPIFFS_CFG_LOG_PAGE_SZ(fs))
#define spiffs_get_cache(fs) \
((spiffs_cache *)((fs)->cache))
#define spiffs_get_cache_page_hdr(fs, c, ix) \
((spiffs_cache_page *)(&((c)->cpages[(ix) * SPIFFS_CACHE_PAGE_SIZE(fs)])))
#define spiffs_get_cache_page(fs, c, ix) \
((u8_t *)(&((c)->cpages[(ix) * SPIFFS_CACHE_PAGE_SIZE(fs)])) + sizeof(spiffs_cache_page))
// cache page struct
typedef struct {
// cache flags
u8_t flags;
// cache page index
u8_t ix;
// last access of this cache page
u32_t last_access;
union {
// type read cache
struct {
// read cache page index
spiffs_page_ix pix;
};
#if SPIFFS_CACHE_WR
// type write cache
struct {
// write cache
spiffs_obj_id obj_id;
// offset in cache page
u32_t offset;
// size of cache page
u16_t size;
};
#endif
};
} spiffs_cache_page;
// cache struct
typedef struct {
u8_t cpage_count;
u32_t last_access;
u32_t cpage_use_map;
u32_t cpage_use_mask;
u8_t *cpages;
} spiffs_cache;
#endif
// spiffs nucleus file descriptor
typedef struct {
// the filesystem of this descriptor
spiffs *fs;
// number of file descriptor - if 0, the file descriptor is closed
spiffs_file file_nbr;
// object id - if SPIFFS_OBJ_ID_ERASED, the file was deleted
spiffs_obj_id obj_id;
// size of the file
u32_t size;
// cached object index header page index
spiffs_page_ix objix_hdr_pix;
// cached offset object index page index
spiffs_page_ix cursor_objix_pix;
// cached offset object index span index
spiffs_span_ix cursor_objix_spix;
// current absolute offset
u32_t offset;
// current file descriptor offset (cached)
u32_t fdoffset;
// fd flags
spiffs_flags flags;
#if SPIFFS_CACHE_WR
spiffs_cache_page *cache_page;
#endif
#if SPIFFS_TEMPORAL_FD_CACHE
// djb2 hash of filename
u32_t name_hash;
// hit score (score == 0 indicates never used fd)
u16_t score;
#endif
#if SPIFFS_IX_MAP
// spiffs index map, if 0 it means unmapped
spiffs_ix_map *ix_map;
#endif
} spiffs_fd;
// object structs
// page header, part of each page except object lookup pages
// NB: this is always aligned when the data page is an object index,
// as in this case struct spiffs_page_object_ix is used
typedef struct SPIFFS_PACKED {
// object id
spiffs_obj_id obj_id;
// object span index
spiffs_span_ix span_ix;
// flags
u8_t flags;
} spiffs_page_header;
// object index header page header
typedef struct SPIFFS_PACKED
#if SPIFFS_ALIGNED_OBJECT_INDEX_TABLES
__attribute(( aligned(sizeof(spiffs_page_ix)) ))
#endif
{
// common page header
spiffs_page_header p_hdr;
// alignment
u8_t _align[4 - ((sizeof(spiffs_page_header)&3)==0 ? 4 : (sizeof(spiffs_page_header)&3))];
// size of object
u32_t size;
// type of object
spiffs_obj_type type;
// name of object
u8_t name[SPIFFS_OBJ_NAME_LEN];
#if SPIFFS_OBJ_META_LEN
// metadata. not interpreted by SPIFFS in any way.
u8_t meta[SPIFFS_OBJ_META_LEN];
#endif
} spiffs_page_object_ix_header;
// object index page header
typedef struct SPIFFS_PACKED {
spiffs_page_header p_hdr;
u8_t _align[4 - ((sizeof(spiffs_page_header)&3)==0 ? 4 : (sizeof(spiffs_page_header)&3))];
} spiffs_page_object_ix;
// callback func for object lookup visitor
typedef s32_t (*spiffs_visitor_f)(spiffs *fs, spiffs_obj_id id, spiffs_block_ix bix, int ix_entry,
const void *user_const_p, void *user_var_p);
#if SPIFFS_CACHE
#define _spiffs_rd(fs, op, fh, addr, len, dst) \
spiffs_phys_rd((fs), (op), (fh), (addr), (len), (dst))
#define _spiffs_wr(fs, op, fh, addr, len, src) \
spiffs_phys_wr((fs), (op), (fh), (addr), (len), (src))
#else
#define _spiffs_rd(fs, op, fh, addr, len, dst) \
spiffs_phys_rd((fs), (addr), (len), (dst))
#define _spiffs_wr(fs, op, fh, addr, len, src) \
spiffs_phys_wr((fs), (addr), (len), (src))
#endif
#ifndef MIN
#define MIN(a,b) ((a) < (b) ? (a) : (b))
#endif
#ifndef MAX
#define MAX(a,b) ((a) > (b) ? (a) : (b))
#endif
// ---------------
s32_t spiffs_phys_rd(
spiffs *fs,
#if SPIFFS_CACHE
u8_t op,
spiffs_file fh,
#endif
u32_t addr,
u32_t len,
u8_t *dst);
s32_t spiffs_phys_wr(
spiffs *fs,
#if SPIFFS_CACHE
u8_t op,
spiffs_file fh,
#endif
u32_t addr,
u32_t len,
u8_t *src);
s32_t spiffs_phys_cpy(
spiffs *fs,
spiffs_file fh,
u32_t dst,
u32_t src,
u32_t len);
s32_t spiffs_phys_count_free_blocks(
spiffs *fs);
s32_t spiffs_obj_lu_find_entry_visitor(
spiffs *fs,
spiffs_block_ix starting_block,
int starting_lu_entry,
u8_t flags,
spiffs_obj_id obj_id,
spiffs_visitor_f v,
const void *user_const_p,
void *user_var_p,
spiffs_block_ix *block_ix,
int *lu_entry);
s32_t spiffs_erase_block(
spiffs *fs,
spiffs_block_ix bix);
#if SPIFFS_USE_MAGIC && SPIFFS_USE_MAGIC_LENGTH
s32_t spiffs_probe(
spiffs_config *cfg);
#endif // SPIFFS_USE_MAGIC && SPIFFS_USE_MAGIC_LENGTH
// ---------------
s32_t spiffs_obj_lu_scan(
spiffs *fs);
s32_t spiffs_obj_lu_find_free_obj_id(
spiffs *fs,
spiffs_obj_id *obj_id,
const u8_t *conflicting_name);
s32_t spiffs_obj_lu_find_free(
spiffs *fs,
spiffs_block_ix starting_block,
int starting_lu_entry,
spiffs_block_ix *block_ix,
int *lu_entry);
s32_t spiffs_obj_lu_find_id(
spiffs *fs,
spiffs_block_ix starting_block,
int starting_lu_entry,
spiffs_obj_id obj_id,
spiffs_block_ix *block_ix,
int *lu_entry);
s32_t spiffs_obj_lu_find_id_and_span(
spiffs *fs,
spiffs_obj_id obj_id,
spiffs_span_ix spix,
spiffs_page_ix exclusion_pix,
spiffs_page_ix *pix);
s32_t spiffs_obj_lu_find_id_and_span_by_phdr(
spiffs *fs,
spiffs_obj_id obj_id,
spiffs_span_ix spix,
spiffs_page_ix exclusion_pix,
spiffs_page_ix *pix);
// ---------------
s32_t spiffs_page_allocate_data(
spiffs *fs,
spiffs_obj_id obj_id,
spiffs_page_header *ph,
u8_t *data,
u32_t len,
u32_t page_offs,
u8_t finalize,
spiffs_page_ix *pix);
s32_t spiffs_page_move(
spiffs *fs,
spiffs_file fh,
u8_t *page_data,
spiffs_obj_id obj_id,
spiffs_page_header *page_hdr,
spiffs_page_ix src_pix,
spiffs_page_ix *dst_pix);
s32_t spiffs_page_delete(
spiffs *fs,
spiffs_page_ix pix);
// ---------------
s32_t spiffs_object_create(
spiffs *fs,
spiffs_obj_id obj_id,
const u8_t name[],
const u8_t meta[],
spiffs_obj_type type,
spiffs_page_ix *objix_hdr_pix);
s32_t spiffs_object_update_index_hdr(
spiffs *fs,
spiffs_fd *fd,
spiffs_obj_id obj_id,
spiffs_page_ix objix_hdr_pix,
u8_t *new_objix_hdr_data,
const u8_t name[],
const u8_t meta[],
u32_t size,
spiffs_page_ix *new_pix);
#if SPIFFS_IX_MAP
s32_t spiffs_populate_ix_map(
spiffs *fs,
spiffs_fd *fd,
u32_t vec_entry_start,
u32_t vec_entry_end);
#endif
void spiffs_cb_object_event(
spiffs *fs,
spiffs_page_object_ix *objix,
int ev,
spiffs_obj_id obj_id,
spiffs_span_ix spix,
spiffs_page_ix new_pix,
u32_t new_size);
s32_t spiffs_object_open_by_id(
spiffs *fs,
spiffs_obj_id obj_id,
spiffs_fd *f,
spiffs_flags flags,
spiffs_mode mode);
s32_t spiffs_object_open_by_page(
spiffs *fs,
spiffs_page_ix pix,
spiffs_fd *f,
spiffs_flags flags,
spiffs_mode mode);
s32_t spiffs_object_append(
spiffs_fd *fd,
u32_t offset,
u8_t *data,
u32_t len);
s32_t spiffs_object_modify(
spiffs_fd *fd,
u32_t offset,
u8_t *data,
u32_t len);
s32_t spiffs_object_read(
spiffs_fd *fd,
u32_t offset,
u32_t len,
u8_t *dst);
s32_t spiffs_object_truncate(
spiffs_fd *fd,
u32_t new_len,
u8_t remove_object);
s32_t spiffs_object_find_object_index_header_by_name(
spiffs *fs,
const u8_t name[SPIFFS_OBJ_NAME_LEN],
spiffs_page_ix *pix);
// ---------------
s32_t spiffs_gc_check(
spiffs *fs,
u32_t len);
s32_t spiffs_gc_erase_page_stats(
spiffs *fs,
spiffs_block_ix bix);
s32_t spiffs_gc_find_candidate(
spiffs *fs,
spiffs_block_ix **block_candidate,
int *candidate_count,
char fs_crammed);
s32_t spiffs_gc_clean(
spiffs *fs,
spiffs_block_ix bix);
s32_t spiffs_gc_quick(
spiffs *fs, u16_t max_free_pages);
// ---------------
s32_t spiffs_fd_find_new(
spiffs *fs,
spiffs_fd **fd,
const char *name);
s32_t spiffs_fd_return(
spiffs *fs,
spiffs_file f);
s32_t spiffs_fd_get(
spiffs *fs,
spiffs_file f,
spiffs_fd **fd);
#if SPIFFS_TEMPORAL_FD_CACHE
void spiffs_fd_temporal_cache_rehash(
spiffs *fs,
const char *old_path,
const char *new_path);
#endif
#if SPIFFS_CACHE
void spiffs_cache_init(
spiffs *fs);
void spiffs_cache_drop_page(
spiffs *fs,
spiffs_page_ix pix);
#if SPIFFS_CACHE_WR
spiffs_cache_page *spiffs_cache_page_allocate_by_fd(
spiffs *fs,
spiffs_fd *fd);
void spiffs_cache_fd_release(
spiffs *fs,
spiffs_cache_page *cp);
spiffs_cache_page *spiffs_cache_page_get_by_fd(
spiffs *fs,
spiffs_fd *fd);
#endif
#endif
s32_t spiffs_lookup_consistency_check(
spiffs *fs,
u8_t check_all_objects);
s32_t spiffs_page_consistency_check(
spiffs *fs);
s32_t spiffs_object_index_consistency_check(
spiffs *fs);
// memcpy macro,
// checked in test builds, otherwise plain memcpy (unless already defined)
#ifdef _SPIFFS_TEST
#define _SPIFFS_MEMCPY(__d, __s, __l) do { \
intptr_t __a1 = (intptr_t)((u8_t*)(__s)); \
intptr_t __a2 = (intptr_t)((u8_t*)(__s)+(__l)); \
intptr_t __b1 = (intptr_t)((u8_t*)(__d)); \
intptr_t __b2 = (intptr_t)((u8_t*)(__d)+(__l)); \
if (__a1 <= __b2 && __b1 <= __a2) { \
printf("FATAL OVERLAP: memcpy from %lx..%lx to %lx..%lx\n", __a1, __a2, __b1, __b2); \
ERREXIT(); \
} \
memcpy((__d),(__s),(__l)); \
} while (0)
#else
#ifndef _SPIFFS_MEMCPY
#define _SPIFFS_MEMCPY(__d, __s, __l) do{memcpy((__d),(__s),(__l));}while(0)
#endif
#endif //_SPIFFS_TEST
#endif /* SPIFFS_NUCLEUS_H_ */

12
src/spiffs/test/main.c
Unescape View File

@ -0,0 +1,12 @@
#include <stdlib.h>
#ifndef NO_TEST
#include "testrunner.h"
#endif
int main(int argc, char **args) {
#ifndef NO_TEST
run_tests(argc, args);
#endif
exit(EXIT_SUCCESS);
}

84
src/spiffs/test/params_test.h
Unescape View File

@ -0,0 +1,84 @@
/*
* params_test.h
*
* Created on: May 26, 2013
* Author: petera
*/
#ifndef PARAMS_TEST_H_
#define PARAMS_TEST_H_
//////////////// TEST PARAMS ////////////////
// default test total emulated spi flash size
#define PHYS_FLASH_SIZE (16*1024*1024)
// default test spiffs file system size
#define SPIFFS_FLASH_SIZE (2*1024*1024)
// default test spiffs file system offset in emulated spi flash
#define SPIFFS_PHYS_ADDR (4*1024*1024)
// default test sector size
#define SECTOR_SIZE 65536
// default test logical block size
#define LOG_BLOCK (SECTOR_SIZE*2)
// default test logical page size
#define LOG_PAGE (SECTOR_SIZE/256)
// default test number of filedescs
#define DEFAULT_NUM_FD 16
// default test number of cache pages
#define DEFAULT_NUM_CACHE_PAGES 8
// When testing, test bench create reference files for comparison on
// the actual hard drive. By default, put these on ram drive for speed.
#define TEST_PATH "/dev/shm/spiffs/test-data/"
#define ASSERT(c, m) real_assert((c),(m), __FILE__, __LINE__);
void real_assert(int c, const char *n, const char *file, int l);
/////////// SPIFFS BUILD CONFIG ////////////
// test using filesystem magic
#ifndef SPIFFS_USE_MAGIC
#define SPIFFS_USE_MAGIC 1
#endif
// test using filesystem magic length
#ifndef SPIFFS_USE_MAGIC_LENGTH
#define SPIFFS_USE_MAGIC_LENGTH 1
#endif
// test using extra param in callback
#ifndef SPIFFS_HAL_CALLBACK_EXTRA
#define SPIFFS_HAL_CALLBACK_EXTRA 1
#endif
// test using filehandle offset
#ifndef SPIFFS_FILEHDL_OFFSET
#define SPIFFS_FILEHDL_OFFSET 1
// use this offset
#define TEST_SPIFFS_FILEHDL_OFFSET 0x1000
#endif
#ifdef NO_TEST
#define SPIFFS_LOCK(fs)
#define SPIFFS_UNLOCK(fs)
#else
struct spiffs_t;
extern void test_lock(struct spiffs_t *fs);
extern void test_unlock(struct spiffs_t *fs);
#define SPIFFS_LOCK(fs) test_lock(fs)
#define SPIFFS_UNLOCK(fs) test_unlock(fs)
#endif
// dbg output
#define SPIFFS_DBG(_f, ...) //printf("\x1b[32m" _f "\x1b[0m", ## __VA_ARGS__)
#define SPIFFS_API_DBG(_f, ...) //printf("\n\x1b[1m\x1b[7m" _f "\x1b[0m", ## __VA_ARGS__)
#define SPIFFS_GC_DBG(_f, ...) //printf("\x1b[36m" _f "\x1b[0m", ## __VA_ARGS__)
#define SPIFFS_CACHE_DBG(_f, ...) //printf("\x1b[33m" _f "\x1b[0m", ## __VA_ARGS__)
#define SPIFFS_CHECK_DBG(_f, ...) //printf("\x1b[31m" _f "\x1b[0m", ## __VA_ARGS__)
// needed types
typedef signed int s32_t;
typedef unsigned int u32_t;
typedef signed short s16_t;
typedef unsigned short u16_t;
typedef signed char s8_t;
typedef unsigned char u8_t;
#endif /* PARAMS_TEST_H_ */

1266
src/spiffs/test/test_bugreports.c
View File

File diff suppressed because it is too large Load Diff

427
src/spiffs/test/test_check.c
Unescape View File

@ -0,0 +1,427 @@
/*
* test_dev.c
*
* Created on: Jul 14, 2013
* Author: petera
*/
#include "testrunner.h"
#include "test_spiffs.h"
#include "spiffs_nucleus.h"
#include "spiffs.h"
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <dirent.h>
#include <unistd.h>
SUITE(check_tests)
static void setup() {
_setup();
}
static void teardown() {
_teardown();
}
TEST(evil_write) {
fs_set_validate_flashing(0);
printf("writing corruption to block 1 data range (leaving lu intact)\n");
u32_t data_range = SPIFFS_CFG_LOG_BLOCK_SZ(FS) -
SPIFFS_CFG_LOG_PAGE_SZ(FS) * (SPIFFS_OBJ_LOOKUP_PAGES(FS));
u8_t *corruption = malloc(data_range);
memrand(corruption, data_range);
u32_t addr = 0 * SPIFFS_CFG_LOG_PAGE_SZ(FS) * SPIFFS_OBJ_LOOKUP_PAGES(FS);
area_write(addr, corruption, data_range);
free(corruption);
int size = SPIFFS_DATA_PAGE_SIZE(FS)*3;
int res = test_create_and_write_file("file", size, size);
printf("CHECK1-----------------\n");
SPIFFS_check(FS);
printf("CHECK2-----------------\n");
SPIFFS_check(FS);
printf("CHECK3-----------------\n");
SPIFFS_check(FS);
res = test_create_and_write_file("file2", size, size);
TEST_CHECK(res >= 0);
return TEST_RES_OK;
} TEST_END
TEST(lu_check1) {
int size = SPIFFS_DATA_PAGE_SIZE(FS)*3;
int res = test_create_and_write_file("file", size, size);
TEST_CHECK(res >= 0);
res = read_and_verify("file");
TEST_CHECK(res >= 0);
spiffs_file fd = SPIFFS_open(FS, "file", SPIFFS_RDONLY, 0);
TEST_CHECK(fd > 0);
spiffs_stat s;
res = SPIFFS_fstat(FS, fd, &s);
TEST_CHECK(res >= 0);
SPIFFS_close(FS, fd);
// modify lu entry data page index 1
spiffs_page_ix pix;
res = spiffs_obj_lu_find_id_and_span(FS, s.obj_id & ~SPIFFS_OBJ_ID_IX_FLAG, 1, 0, &pix);
TEST_CHECK(res >= 0);
// reset lu entry to being erased, but keep page data
spiffs_obj_id obj_id = SPIFFS_OBJ_ID_DELETED;
spiffs_block_ix bix = SPIFFS_BLOCK_FOR_PAGE(FS, pix);
int entry = SPIFFS_OBJ_LOOKUP_ENTRY_FOR_PAGE(FS, pix);
u32_t addr = SPIFFS_BLOCK_TO_PADDR(FS, bix) + entry*sizeof(spiffs_obj_id);
area_write(addr, (u8_t*)&obj_id, sizeof(spiffs_obj_id));
#if SPIFFS_CACHE
spiffs_cache *cache = spiffs_get_cache(FS);
cache->cpage_use_map = 0;
#endif
SPIFFS_check(FS);
return TEST_RES_OK;
} TEST_END
TEST(page_cons1) {
int size = SPIFFS_DATA_PAGE_SIZE(FS)*3;
int res = test_create_and_write_file("file", size, size);
TEST_CHECK(res >= 0);
res = read_and_verify("file");
TEST_CHECK(res >= 0);
spiffs_file fd = SPIFFS_open(FS, "file", SPIFFS_RDONLY, 0);
TEST_CHECK(fd > 0);
spiffs_stat s;
res = SPIFFS_fstat(FS, fd, &s);
TEST_CHECK(res >= 0);
SPIFFS_close(FS, fd);
// modify object index, find object index header
spiffs_page_ix pix;
res = spiffs_obj_lu_find_id_and_span(FS, s.obj_id | SPIFFS_OBJ_ID_IX_FLAG, 0, 0, &pix);
TEST_CHECK(res >= 0);
// set object index entry 2 to a bad page
u32_t addr = SPIFFS_PAGE_TO_PADDR(FS, pix) + sizeof(spiffs_page_object_ix_header) + 0 * sizeof(spiffs_page_ix);
spiffs_page_ix bad_pix_ref = 0x55;
area_write(addr, (u8_t*)&bad_pix_ref, sizeof(spiffs_page_ix));
area_write(addr + sizeof(spiffs_page_ix), (u8_t*)&bad_pix_ref, sizeof(spiffs_page_ix));
// delete all cache
#if SPIFFS_CACHE
spiffs_cache *cache = spiffs_get_cache(FS);
cache->cpage_use_map = 0;
#endif
SPIFFS_check(FS);
res = read_and_verify("file");
TEST_CHECK(res >= 0);
return TEST_RES_OK;
} TEST_END
TEST(page_cons2) {
int size = SPIFFS_DATA_PAGE_SIZE(FS)*3;
int res = test_create_and_write_file("file", size, size);
TEST_CHECK(res >= 0);
res = read_and_verify("file");
TEST_CHECK(res >= 0);
spiffs_file fd = SPIFFS_open(FS, "file", SPIFFS_RDONLY, 0);
TEST_CHECK(fd > 0);
spiffs_stat s;
res = SPIFFS_fstat(FS, fd, &s);
TEST_CHECK(res >= 0);
SPIFFS_close(FS, fd);
// modify object index, find object index header
spiffs_page_ix pix;
res = spiffs_obj_lu_find_id_and_span(FS, s.obj_id | SPIFFS_OBJ_ID_IX_FLAG, 0, 0, &pix);
TEST_CHECK(res >= 0);
// find data page span index 0
spiffs_page_ix dpix;
res = spiffs_obj_lu_find_id_and_span(FS, s.obj_id & ~SPIFFS_OBJ_ID_IX_FLAG, 0, 0, &dpix);
TEST_CHECK(res >= 0);
// set object index entry 1+2 to a data page 0
u32_t addr = SPIFFS_PAGE_TO_PADDR(FS, pix) + sizeof(spiffs_page_object_ix_header) + 1 * sizeof(spiffs_page_ix);
spiffs_page_ix bad_pix_ref = dpix;
area_write(addr, (u8_t*)&bad_pix_ref, sizeof(spiffs_page_ix));
area_write(addr+sizeof(spiffs_page_ix), (u8_t*)&bad_pix_ref, sizeof(spiffs_page_ix));
// delete all cache
#if SPIFFS_CACHE
spiffs_cache *cache = spiffs_get_cache(FS);
cache->cpage_use_map = 0;
#endif
SPIFFS_check(FS);
res = read_and_verify("file");
TEST_CHECK(res >= 0);
return TEST_RES_OK;
} TEST_END
TEST(page_cons3) {
int size = SPIFFS_DATA_PAGE_SIZE(FS)*3;
int res = test_create_and_write_file("file", size, size);
TEST_CHECK(res >= 0);
res = read_and_verify("file");
TEST_CHECK(res >= 0);
spiffs_file fd = SPIFFS_open(FS, "file", SPIFFS_RDONLY, 0);
TEST_CHECK(fd > 0);
spiffs_stat s;
res = SPIFFS_fstat(FS, fd, &s);
TEST_CHECK(res >= 0);
SPIFFS_close(FS, fd);
// modify object index, find object index header
spiffs_page_ix pix;
res = spiffs_obj_lu_find_id_and_span(FS, s.obj_id | SPIFFS_OBJ_ID_IX_FLAG, 0, 0, &pix);
TEST_CHECK(res >= 0);
// set object index entry 1+2 lookup page
u32_t addr = SPIFFS_PAGE_TO_PADDR(FS, pix) + sizeof(spiffs_page_object_ix_header) + 1 * sizeof(spiffs_page_ix);
spiffs_page_ix bad_pix_ref = SPIFFS_PAGES_PER_BLOCK(FS) * (*FS.block_count - 2);
area_write(addr, (u8_t*)&bad_pix_ref, sizeof(spiffs_page_ix));
area_write(addr+sizeof(spiffs_page_ix), (u8_t*)&bad_pix_ref, sizeof(spiffs_page_ix));
// delete all cache
#if SPIFFS_CACHE
spiffs_cache *cache = spiffs_get_cache(FS);
cache->cpage_use_map = 0;
#endif
SPIFFS_check(FS);
res = read_and_verify("file");
TEST_CHECK(res >= 0);
return TEST_RES_OK;
} TEST_END
TEST(page_cons_final) {
int size = SPIFFS_DATA_PAGE_SIZE(FS)*3;
int res = test_create_and_write_file("file", size, size);
TEST_CHECK(res >= 0);
res = read_and_verify("file");
TEST_CHECK(res >= 0);
spiffs_file fd = SPIFFS_open(FS, "file", SPIFFS_RDONLY, 0);
TEST_CHECK(fd > 0);
spiffs_stat s;
res = SPIFFS_fstat(FS, fd, &s);
TEST_CHECK(res >= 0);
SPIFFS_close(FS, fd);
// modify page header, make unfinalized
spiffs_page_ix pix;
res = spiffs_obj_lu_find_id_and_span(FS, s.obj_id & ~SPIFFS_OBJ_ID_IX_FLAG, 1, 0, &pix);
TEST_CHECK(res >= 0);
// set page span ix 1 as unfinalized
u32_t addr = SPIFFS_PAGE_TO_PADDR(FS, pix) + offsetof(spiffs_page_header, flags);
u8_t flags;
area_read(addr, (u8_t*)&flags, 1);
flags |= SPIFFS_PH_FLAG_FINAL;
area_write(addr, (u8_t*)&flags, 1);
// delete all cache
#if SPIFFS_CACHE
spiffs_cache *cache = spiffs_get_cache(FS);
cache->cpage_use_map = 0;
#endif
SPIFFS_check(FS);
res = read_and_verify("file");
TEST_CHECK(res >= 0);
return TEST_RES_OK;
} TEST_END
TEST(index_cons1) {
int size = SPIFFS_DATA_PAGE_SIZE(FS)*SPIFFS_PAGES_PER_BLOCK(FS);
int res = test_create_and_write_file("file", size, size);
TEST_CHECK(res >= 0);
res = read_and_verify("file");
TEST_CHECK(res >= 0);
spiffs_file fd = SPIFFS_open(FS, "file", SPIFFS_RDONLY, 0);
TEST_CHECK(fd > 0);
spiffs_stat s;
res = SPIFFS_fstat(FS, fd, &s);
TEST_CHECK(res >= 0);
SPIFFS_close(FS, fd);
// modify lu entry data page index header
spiffs_page_ix pix;
res = spiffs_obj_lu_find_id_and_span(FS, s.obj_id | SPIFFS_OBJ_ID_IX_FLAG, 0, 0, &pix);
TEST_CHECK(res >= 0);
printf(" deleting lu entry pix %04x\n", pix);
// reset lu entry to being erased, but keep page data
spiffs_obj_id obj_id = SPIFFS_OBJ_ID_DELETED;
spiffs_block_ix bix = SPIFFS_BLOCK_FOR_PAGE(FS, pix);
int entry = SPIFFS_OBJ_LOOKUP_ENTRY_FOR_PAGE(FS, pix);
u32_t addr = SPIFFS_BLOCK_TO_PADDR(FS, bix) + entry * sizeof(spiffs_obj_id);
area_write(addr, (u8_t*)&obj_id, sizeof(spiffs_obj_id));
#if SPIFFS_CACHE
spiffs_cache *cache = spiffs_get_cache(FS);
cache->cpage_use_map = 0;
#endif
SPIFFS_check(FS);
res = read_and_verify("file");
TEST_CHECK(res >= 0);
return TEST_RES_OK;
} TEST_END
TEST(index_cons2) {
int size = SPIFFS_DATA_PAGE_SIZE(FS)*SPIFFS_PAGES_PER_BLOCK(FS);
int res = test_create_and_write_file("file", size, size);
TEST_CHECK(res >= 0);
res = read_and_verify("file");
TEST_CHECK(res >= 0);
spiffs_file fd = SPIFFS_open(FS, "file", SPIFFS_RDONLY, 0);
TEST_CHECK(fd > 0);
spiffs_stat s;
res = SPIFFS_fstat(FS, fd, &s);
TEST_CHECK(res >= 0);
SPIFFS_close(FS, fd);
// modify lu entry data page index header
spiffs_page_ix pix;
res = spiffs_obj_lu_find_id_and_span(FS, s.obj_id | SPIFFS_OBJ_ID_IX_FLAG, 0, 0, &pix);
TEST_CHECK(res >= 0);
printf(" writing lu entry for index page, ix %04x, as data page\n", pix);
spiffs_obj_id obj_id = 0x1234;
spiffs_block_ix bix = SPIFFS_BLOCK_FOR_PAGE(FS, pix);
int entry = SPIFFS_OBJ_LOOKUP_ENTRY_FOR_PAGE(FS, pix);
u32_t addr = SPIFFS_BLOCK_TO_PADDR(FS, bix) + entry * sizeof(spiffs_obj_id);
area_write(addr, (u8_t*)&obj_id, sizeof(spiffs_obj_id));
#if SPIFFS_CACHE
spiffs_cache *cache = spiffs_get_cache(FS);
cache->cpage_use_map = 0;
#endif
SPIFFS_check(FS);
res = read_and_verify("file");
TEST_CHECK(res >= 0);
return TEST_RES_OK;
} TEST_END
TEST(index_cons3) {
int size = SPIFFS_DATA_PAGE_SIZE(FS)*SPIFFS_PAGES_PER_BLOCK(FS);
int res = test_create_and_write_file("file", size, size);
TEST_CHECK(res >= 0);
res = read_and_verify("file");
TEST_CHECK(res >= 0);
spiffs_file fd = SPIFFS_open(FS, "file", SPIFFS_RDONLY, 0);
TEST_CHECK(fd > 0);
spiffs_stat s;
res = SPIFFS_fstat(FS, fd, &s);
TEST_CHECK(res >= 0);
SPIFFS_close(FS, fd);
// modify lu entry data page index header
spiffs_page_ix pix;
res = spiffs_obj_lu_find_id_and_span(FS, s.obj_id | SPIFFS_OBJ_ID_IX_FLAG, 0, 0, &pix);
TEST_CHECK(res >= 0);
printf(" setting lu entry pix %04x to another index page\n", pix);
// reset lu entry to being erased, but keep page data
spiffs_obj_id obj_id = 1234 | SPIFFS_OBJ_ID_IX_FLAG;
spiffs_block_ix bix = SPIFFS_BLOCK_FOR_PAGE(FS, pix);
int entry = SPIFFS_OBJ_LOOKUP_ENTRY_FOR_PAGE(FS, pix);
u32_t addr = SPIFFS_BLOCK_TO_PADDR(FS, bix) + entry * sizeof(spiffs_obj_id);
area_write(addr, (u8_t*)&obj_id, sizeof(spiffs_obj_id));
#if SPIFFS_CACHE
spiffs_cache *cache = spiffs_get_cache(FS);
cache->cpage_use_map = 0;
#endif
SPIFFS_check(FS);
res = read_and_verify("file");
TEST_CHECK(res >= 0);
return TEST_RES_OK;
} TEST_END
TEST(index_cons4) {
int size = SPIFFS_DATA_PAGE_SIZE(FS)*SPIFFS_PAGES_PER_BLOCK(FS);
int res = test_create_and_write_file("file", size, size);
TEST_CHECK(res >= 0);
res = read_and_verify("file");
TEST_CHECK(res >= 0);
spiffs_file fd = SPIFFS_open(FS, "file", SPIFFS_RDONLY, 0);
TEST_CHECK(fd > 0);
spiffs_stat s;
res = SPIFFS_fstat(FS, fd, &s);
TEST_CHECK(res >= 0);
SPIFFS_close(FS, fd);
// modify lu entry data page index header, flags
spiffs_page_ix pix;
res = spiffs_obj_lu_find_id_and_span(FS, s.obj_id | SPIFFS_OBJ_ID_IX_FLAG, 0, 0, &pix);
TEST_CHECK(res >= 0);
printf(" cue objix hdr deletion in page %04x\n", pix);
// set flags as deleting ix header
u32_t addr = SPIFFS_PAGE_TO_PADDR(FS, pix) + offsetof(spiffs_page_header, flags);
u8_t flags = 0xff & ~(SPIFFS_PH_FLAG_FINAL | SPIFFS_PH_FLAG_USED | SPIFFS_PH_FLAG_INDEX | SPIFFS_PH_FLAG_IXDELE);
area_write(addr, (u8_t*)&flags, 1);
#if SPIFFS_CACHE
spiffs_cache *cache = spiffs_get_cache(FS);
cache->cpage_use_map = 0;
#endif
SPIFFS_check(FS);
return TEST_RES_OK;
} TEST_END
SUITE_TESTS(check_tests)
ADD_TEST(evil_write)
ADD_TEST(lu_check1)
ADD_TEST(page_cons1)
ADD_TEST(page_cons2)
ADD_TEST(page_cons3)
ADD_TEST(page_cons_final)
ADD_TEST(index_cons1)
ADD_TEST(index_cons2)
ADD_TEST(index_cons3)
ADD_TEST(index_cons4)
SUITE_END(check_tests)

122
src/spiffs/test/test_dev.c
Unescape View File

@ -0,0 +1,122 @@
/*
* test_dev.c
*
* Created on: Jul 14, 2013
* Author: petera
*/
#include "testrunner.h"
#include "test_spiffs.h"
#include "spiffs_nucleus.h"
#include "spiffs.h"
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <dirent.h>
#include <unistd.h>
SUITE(dev_tests)
static void setup() {
_setup();
}
static void teardown() {
_teardown();
}
TEST(interrupted_write) {
char *name = "interrupt";
char *name2 = "interrupt2";
int res;
spiffs_file fd;
const u32_t sz = SPIFFS_CFG_LOG_PAGE_SZ(FS)*8;
u8_t *buf = malloc(sz);
memrand(buf, sz);
printf(" create reference file\n");
fd = SPIFFS_open(FS, name, SPIFFS_RDWR | SPIFFS_CREAT | SPIFFS_TRUNC, 0);
TEST_CHECK(fd > 0);
clear_flash_ops_log();
res = SPIFFS_write(FS, fd, buf, sz);
TEST_CHECK(res >= 0);
SPIFFS_close(FS, fd);
u32_t written = get_flash_ops_log_write_bytes();
printf(" written bytes: %i\n", written);
printf(" create error file\n");
fd = SPIFFS_open(FS, name2, SPIFFS_RDWR | SPIFFS_CREAT | SPIFFS_TRUNC, 0);
TEST_CHECK(fd > 0);
clear_flash_ops_log();
invoke_error_after_write_bytes(written/2, 0);
res = SPIFFS_write(FS, fd, buf, sz);
SPIFFS_close(FS, fd);
TEST_CHECK(SPIFFS_errno(FS) == SPIFFS_ERR_TEST);
clear_flash_ops_log();
#if SPIFFS_CACHE
// delete all cache
spiffs_cache *cache = spiffs_get_cache(FS);
cache->cpage_use_map = 0;
#endif
printf(" read error file\n");
fd = SPIFFS_open(FS, name2, SPIFFS_RDONLY, 0);
TEST_CHECK(fd > 0);
spiffs_stat s;
res = SPIFFS_fstat(FS, fd, &s);
TEST_CHECK(res >= 0);
printf(" file size: %i\n", s.size);
if (s.size > 0) {
u8_t *buf2 = malloc(s.size);
res = SPIFFS_read(FS, fd, buf2, s.size);
TEST_CHECK(res >= 0);
u32_t ix = 0;
for (ix = 0; ix < s.size; ix += 16) {
int i;
printf(" ");
for (i = 0; i < 16; i++) {
printf("%02x", buf[ix+i]);
}
printf(" ");
for (i = 0; i < 16; i++) {
printf("%02x", buf2[ix+i]);
}
printf("\n");
}
free(buf2);
}
SPIFFS_close(FS, fd);
printf(" FS check\n");
SPIFFS_check(FS);
printf(" read error file again\n");
fd = SPIFFS_open(FS, name2, SPIFFS_APPEND | SPIFFS_RDWR, 0);
TEST_CHECK(fd > 0);
res = SPIFFS_fstat(FS, fd, &s);
TEST_CHECK(res >= 0);
printf(" file size: %i\n", s.size);
printf(" write file\n");
res = SPIFFS_write(FS, fd, buf, sz);
TEST_CHECK(res >= 0);
SPIFFS_close(FS, fd);
free(buf);
return TEST_RES_OK;
} TEST_END
SUITE_TESTS(dev_tests)
ADD_TEST(interrupted_write)
SUITE_END(dev_tests)

2507
src/spiffs/test/test_hydrogen.c
View File

File diff suppressed because it is too large Load Diff

1114
src/spiffs/test/test_spiffs.c
View File

File diff suppressed because it is too large Load Diff

109
src/spiffs/test/test_spiffs.h
Unescape View File

@ -0,0 +1,109 @@
/*
* test_spiffs.h
*
* Created on: Jun 19, 2013
* Author: petera
*/
#ifndef TEST_SPIFFS_H_
#define TEST_SPIFFS_H_
#include "spiffs.h"
#define FS &__fs
extern spiffs __fs;
#define CHECK(r) if (!(r)) return -1;
#define CHECK_RES(r) if (r < 0) return -1;
#define FS_PURE_DATA_PAGES(fs) \
(SPIFFS_CFG_PHYS_SZ(fs) / SPIFFS_CFG_LOG_PAGE_SZ(fs)- (fs)->block_count * SPIFFS_OBJ_LOOKUP_PAGES(fs))
#define FS_PURE_DATA_SIZE(fs) \
FS_PURE_DATA_PAGES(fs) * SPIFFS_DATA_PAGE_SIZE(fs)
typedef enum {
EMPTY,
SMALL,
MEDIUM,
LARGE,
} tfile_size;
typedef enum {
UNTAMPERED,
APPENDED,
MODIFIED,
REWRITTEN,
} tfile_type;
typedef enum {
SHORT = 3,
NORMAL = 15,
LONG = 100,
} tfile_life;
typedef struct {
tfile_size tsize;
tfile_type ttype;
tfile_life tlife;
} tfile_conf;
typedef struct {
int state;
spiffs_file fd;
tfile_conf cfg;
char name[32];
} tfile;
void fs_reset();
void fs_reset_specific(u32_t addr_offset, u32_t phys_addr, u32_t phys_size,
u32_t phys_sector_size,
u32_t log_block_size, u32_t log_page_size);
s32_t fs_mount_specific(u32_t phys_addr, u32_t phys_size,
u32_t phys_sector_size,
u32_t log_block_size, u32_t log_page_size);
void fs_mount_dump(char *fname,
u32_t addr_offset, u32_t phys_addr, u32_t phys_size,
u32_t phys_sector_size,
u32_t log_block_size, u32_t log_page_size);
void fs_store_dump(char *fname);
void fs_load_dump(char *fname);
void fs_set_addr_offset(u32_t offset);
int read_and_verify(char *name);
int read_and_verify_fd(spiffs_file fd, char *name);
void dump_page(spiffs *fs, spiffs_page_ix p);
void hexdump(u32_t addr, u32_t len);
char *make_test_fname(const char *name);
void clear_test_path();
void area_write(u32_t addr, u8_t *buf, u32_t size);
void area_set(u32_t addr, u8_t d, u32_t size);
void area_read(u32_t addr, u8_t *buf, u32_t size);
void dump_erase_counts(spiffs *fs);
void dump_flash_access_stats();
void set_flash_ops_log(int enable);
void clear_flash_ops_log();
u32_t get_flash_ops_log_read_bytes();
u32_t get_flash_ops_log_write_bytes();
void invoke_error_after_read_bytes(u32_t b, char once_only);
void invoke_error_after_write_bytes(u32_t b, char once_only);
void fs_set_validate_flashing(int i);
int get_error_count();
int count_taken_fds(spiffs *fs);
void memrand(u8_t *b, int len);
int test_create_file(char *name);
int test_create_and_write_file(char *name, int size, int chunk_size);
u32_t get_spiffs_file_crc_by_fd(spiffs_file fd);
u32_t get_spiffs_file_crc(char *name);
void _setup();
void _setup_test_only();
void _teardown();
u32_t tfile_get_size(tfile_size s);
int run_file_config(int cfg_count, tfile_conf* cfgs, int max_runs, int max_concurrent_files, int dbg);
void test_lock(spiffs *fs);
void test_unlock(spiffs *fs);
#endif /* TEST_SPIFFS_H_ */

238
src/spiffs/test/testrunner.c
Unescape View File

@ -0,0 +1,238 @@
/*
* testrunner.c
*
* Created on: Jun 18, 2013
* Author: petera
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <dirent.h>
#include <unistd.h>
#include "testrunner.h"
static struct {
test *tests;
test *_last_test;
int test_count;
void (*on_stop)(test *t);
test_res *failed;
test_res *failed_last;
test_res *stopped;
test_res *stopped_last;
FILE *spec;
char incl_filter[256];
char excl_filter[256];
} test_main;
void test_init(void (*on_stop)(test *t)) {
test_main.on_stop = on_stop;
}
static int abort_on_error = 0;
static int error_count = 0;
static char check_spec(char *name) {
if (test_main.spec) {
fseek(test_main.spec, 0, SEEK_SET);
char *line = NULL;
size_t sz;
ssize_t read;
while ((read = getline(&line, &sz, test_main.spec)) != -1) {
if (strncmp(line, name, strlen(line)-1) == 0) {
free(line);
return 1;
}
}
free(line);
return 0;
} else {
return 1;
}
}
static char check_incl_filter(char *name) {
if (strlen(test_main.incl_filter)== 0) return 1;
return strstr(name, test_main.incl_filter) == 0 ? 0 : 2;
}
static char check_excl_filter(char *name) {
if (strlen(test_main.excl_filter)== 0) return 1;
return strstr(name, test_main.excl_filter) == 0 ? 1 : 0;
}
void _add_test(test_f f, char *name, void (*setup)(test *t), void (*teardown)(test *t), int non_default) {
if (f == 0) return;
if (!check_spec(name)) return;
if (check_incl_filter(name) <= non_default) return;
if (!check_excl_filter(name)) return;
DBGT("adding test %s\n", name);
test *t = malloc(sizeof(test));
memset(t, 0, sizeof(test));
t->f = f;
strcpy(t->name, name);
t->setup = setup;
t->teardown = teardown;
if (test_main.tests == 0) {
test_main.tests = t;
} else {
test_main._last_test->_next = t;
}
test_main._last_test = t;
test_main.test_count++;
}
static void add_res(test *t, test_res **head, test_res **last) {
test_res *tr = malloc(sizeof(test_res));
memset(tr,0,sizeof(test_res));
strcpy(tr->name, t->name);
if (*head == 0) {
*head = tr;
} else {
(*last)->_next = tr;
}
*last = tr;
}
static void dump_res(test_res **head) {
test_res *tr = (*head);
while (tr) {
test_res *next_tr = tr->_next;
printf(" %s\n", tr->name);
free(tr);
tr = next_tr;
}
}
int get_error_count(void) {
return error_count;
}
void inc_error_count(void) {
error_count++;
}
int set_abort_on_error(int val) {
int old_val = abort_on_error;
abort_on_error = val;
return old_val;
}
int get_abort_on_error(void) {
return abort_on_error;
}
int run_tests(int argc, char **args) {
memset(&test_main, 0, sizeof(test_main));
int arg;
int incl_filter = 0;
int excl_filter = 0;
for (arg = 1; arg < argc; arg++) {
if (strlen(args[arg]) == 0) continue;
if (0 == strcmp("-f", args[arg])) {
incl_filter = 1;
continue;
}
if (0 == strcmp("-e", args[arg])) {
excl_filter = 1;
continue;
}
if (incl_filter) {
strcpy(test_main.incl_filter, args[arg]);
incl_filter = 0;
} else if (excl_filter) {
strcpy(test_main.excl_filter, args[arg]);
excl_filter = 0;
} else {
printf("running tests from %s\n", args[arg]);
FILE *fd = fopen(args[1], "r");
if (fd == NULL) {
printf("%s not found\n", args[arg]);
return -2;
}
test_main.spec = fd;
}
}
DBGT("adding suites...\n");
add_suites();
DBGT("%i tests added\n", test_main.test_count);
if (test_main.spec) {
fclose(test_main.spec);
}
if (test_main.test_count == 0) {
printf("No tests to run\n");
return 0;
}
int fd_success = open("_tests_ok", O_APPEND | O_TRUNC | O_CREAT | O_RDWR, S_IRUSR | S_IWUSR);
int fd_bad = open("_tests_fail", O_APPEND | O_TRUNC | O_CREAT | O_RDWR, S_IRUSR | S_IWUSR);
DBGT("running tests...\n");
int ok = 0;
int failed = 0;
int stopped = 0;
test *cur_t = test_main.tests;
int i = 1;
while (cur_t) {
cur_t->setup(cur_t);
test *next_test = cur_t->_next;
DBGT("TEST %i/%i : running test %s\n", i, test_main.test_count, cur_t->name);
i++;
int start_error_count = get_error_count();
int res = cur_t->f(cur_t);
if (res == TEST_RES_OK && get_error_count() != start_error_count) {
res = TEST_RES_FAIL;
}
cur_t->test_result = res;
int fd = res == TEST_RES_OK ? fd_success : fd_bad;
write(fd, cur_t->name, strlen(cur_t->name));
write(fd, "\n", 1);
switch (res) {
case TEST_RES_OK:
ok++;
printf(" .. ok\n");
break;
case TEST_RES_FAIL:
failed++;
printf(" .. FAILED\n");
if (test_main.on_stop) test_main.on_stop(cur_t);
add_res(cur_t, &test_main.failed, &test_main.failed_last);
break;
case TEST_RES_ASSERT:
stopped++;
printf(" .. ABORTED\n");
if (test_main.on_stop) test_main.on_stop(cur_t);
add_res(cur_t, &test_main.stopped, &test_main.stopped_last);
break;
}
cur_t->teardown(cur_t);
free(cur_t);
cur_t = next_test;
}
close(fd_success);
close(fd_bad);
DBGT("ran %i tests\n", test_main.test_count);
printf("Test report, %i tests\n", test_main.test_count);
printf("%i succeeded\n", ok);
printf("%i failed\n", failed);
dump_res(&test_main.failed);
printf("%i stopped\n", stopped);
dump_res(&test_main.stopped);
if (ok < test_main.test_count) {
printf("\nFAILED\n");
return -1;
} else {
printf("\nALL TESTS OK\n");
return 0;
}
}

165
src/spiffs/test/testrunner.h
Unescape View File

@ -0,0 +1,165 @@
/*
* testrunner.h
*
* Created on: Jun 19, 2013
* Author: petera
*/
/*
file mysuite.c:
SUITE(mysuite)
static void setup(test *t) {}
static void teardown(test *t) {}
TEST(mytest) {
printf("mytest runs now..\n");
return 0;
} TEST_END
SUITE_TESTS(mysuite)
ADD_TEST(mytest)
SUITE_END(mysuite)
file mysuite2.c:
SUITE(mysuite2)
static void setup(test *t) {}
static void teardown(test *t) {}
TEST(mytest2a) {
printf("mytest2a runs now..\n");
return 0;
} TEST_END
TEST(mytest2b) {
printf("mytest2b runs now..\n");
return 0;
} TEST_END
SUITE_TESTS(mysuite2)
ADD_TEST(mytest2a)
ADD_TEST(mytest2b)
SUITE_END(mysuite2)
some other file.c:
void add_suites() {
ADD_SUITE(mysuite);
ADD_SUITE(mysuite2);
}
*/
#ifndef TESTRUNNER_H_
#define TESTRUNNER_H_
#define TEST_RES_OK 0
#define TEST_RES_FAIL -1
#define TEST_RES_ASSERT -2
#define ERREXIT() if (get_abort_on_error()) abort(); else inc_error_count()
struct test_s;
typedef int (*test_f)(struct test_s *t);
typedef struct test_s {
test_f f;
char name[256];
void *data;
void (*setup)(struct test_s *t);
void (*teardown)(struct test_s *t);
struct test_s *_next;
unsigned char test_result;
} test;
typedef struct test_res_s {
char name[256];
struct test_res_s *_next;
} test_res;
#define TEST_CHECK(x) if (!(x)) { \
printf(" TEST FAIL %s:%d\n", __FILE__, __LINE__); \
goto __fail_stop; \
}
#define TEST_CHECK_EQ(x, y) if ((x) != (y)) { \
printf(" TEST FAIL %s:%d, %d != %d\n", __FILE__, __LINE__, (int)(x), (int)(y)); \
goto __fail_stop; \
}
#define TEST_CHECK_NEQ(x, y) if ((x) == (y)) { \
printf(" TEST FAIL %s:%d, %d == %d\n", __FILE__, __LINE__, (int)(x), (int)(y)); \
goto __fail_stop; \
}
#define TEST_CHECK_GT(x, y) if ((x) <= (y)) { \
printf(" TEST FAIL %s:%d, %d <= %d\n", __FILE__, __LINE__, (int)(x), (int)(y)); \
goto __fail_stop; \
}
#define TEST_CHECK_LT(x, y) if ((x) >= (y)) { \
printf(" TEST FAIL %s:%d, %d >= %d\n", __FILE__, __LINE__, (int)(x), (int)(y)); \
goto __fail_stop; \
}
#define TEST_CHECK_GE(x, y) if ((x) < (y)) { \
printf(" TEST FAIL %s:%d, %d < %d\n", __FILE__, __LINE__, (int)(x), (int)(y)); \
goto __fail_stop; \
}
#define TEST_CHECK_LE(x, y) if ((x) > (y)) { \
printf(" TEST FAIL %s:%d, %d > %d\n", __FILE__, __LINE__, (int)(x), (int)(y)); \
goto __fail_stop; \
}
#define TEST_ASSERT(x) if (!(x)) { \
printf(" TEST ASSERT %s:%d\n", __FILE__, __LINE__); \
goto __fail_assert; \
}
#define DBGT(...) printf(__VA_ARGS__)
#define str(s) #s
#define SUITE(sui)
#define SUITE_TESTS(sui) \
void _add_suite_tests_##sui(void) {
#define SUITE_END(sui) \
}
#define ADD_TEST(tf) \
_add_test(__test_##tf, str(tf), setup, teardown, 0);
#define ADD_TEST_NON_DEFAULT(tf) \
_add_test(__test_##tf, str(tf), setup, teardown, 1);
#define ADD_SUITE(sui) \
extern void _add_suite_tests_##sui(void); \
_add_suite_tests_##sui();
#define TEST(tf) \
static int __test_##tf(struct test_s *t) { do
#define TEST_END \
while(0); \
__fail_stop: return TEST_RES_FAIL; \
__fail_assert: return TEST_RES_ASSERT; \
}
int set_abort_on_error(int val);
int get_abort_on_error(void);
int get_error_count(void);
void inc_error_count(void);
void add_suites(void);
void test_init(void (*on_stop)(test *t));
// returns 0 if all tests ok, -1 if any test failed, -2 on badness
int run_tests(int argc, char **args);
void _add_suite(const char *suite_name);
void _add_test(test_f f, char *name, void (*setup)(test *t), void (*teardown)(test *t), int non_default);
#endif /* TESTRUNNER_H_ */

15
src/spiffs/test/testsuites.c
Unescape View File

@ -0,0 +1,15 @@
/*
* testsuites.c
*
* Created on: Jun 19, 2013
* Author: petera
*/
#include "testrunner.h"
void add_suites(void) {
//ADD_SUITE(dev_tests);
ADD_SUITE(check_tests);
ADD_SUITE(hydrogen_tests);
ADD_SUITE(bug_tests);
}
Write Preview
Loading…
Cancel
Save