This tutorial describes how to use the pbLua FLASH File System. Files are stored in the on-chip FLASH memory of the microcontroller inside the NXT. You can create, write, and read the files, but more importantly, you can run them as well. You can choose the files manually using a dialog on the NXT LCD, or you can choose it from your program. You can even use the pbLua FLASH File System to log data generated by your robot, and then retrieve it for later analysis.
This tutorial will discuss the pbLua File API and how to use it to create files that you can use for your own robot applications.
As a reminder, there is an excellent online version of Programming in Lua that is a complete description of Lua along with many programming examples, and an even better idea is to purchase the latest Programming in Lua book .
By the end of this tutorial, you'll be using the File API to create text files, read them, and even run them automatically! There's a lot of information in this tutorial - stick with it as the File API unleashes some incredible power that you'll use over and over again.
Using an embedded file system is not quite the same as dealing with the files on your computer. It has long filenames, directories, and many other features that are difficult to bring to a very small embedded microcontroller. The main difference is that we're very "close to the metal" which means that we need to understand a bit about how the FLASH memory in the NXT controller works if we're going to use it effectively.
FLASH is a non-volatile memory technology, which means that it remembers what it was storing even if power is removed from it. This is great news for battery powered robots since it's very easy to have the batteries discharge while running a robot.
Unfortunately, FLASH has a few characteristics that require a little extra work when managing storage:
The FLASH memory on the NXT controller is quite different from a USB memory key, your iPod, or the little chip you put into your digital camera, even though the same basic technilogy is used in all of them. These devices have all kinds of smarts built in that include wear-levelling to avoid burning out individual sectors and require special drivers to work with your computer. These features are simply too complex to implement in the limited memory available on the NXT.
On the bright side, the pbLua File API looks after almost all of the issues in the previous list, so you're not struggling at too low a level either.
If you have done much programming, the first thing you'll notice about the pbLua file system is that it's almost, but not quite, like a standard file system on a computer. Here are the main specifications:
That last item is really important and is due to the way that FLASH memory works on the NXT. Recall that FLASH memory sectors have a limited lifespan. If we kept rewriting the file descriptor block with new filenames and sizes every time we wrote a new file, we'd quickly burn out that sector.
Instead, pbLua takes advantage of special values in the file descriptor to figure out its state.
| Descriptor State | Description |
| Available | The header bits are all ones |
| In Use | The header bits are ones and zeros |
| Erased | The header bits are all zeros |
Recall that FLASH bits can be programmed from one to zero, but not the other way around without going through an erase cycle. To conserve the life of the FLASH in the NXT, erasing a file is really just a matter of clearing all of the bits in that descriptor, but now the descriptor can't be used again - at least not right away.
With all this in mind, let's get down to some exploring of the pbLua File System.
When you first boot up the NXT with pbLua, the FLASH file system is initialized with fresh file descriptors and we're ready to start loading files. Sometimes after upgrading pbLua, the old filesystem may become corrupted. If this happens, and you want to guarantee a clean, fresh, empty filesystem to work with, you can format the FLASH, like this:
-- Format the FLASH file system, erasing all files: nxt.FileFormat(1) -- Format the FLASH file system, moves the file descriptor block to a new -- location, makes erased descriptors usable, leaves exiting files untouched: nxt.FileFormat(0)
You might want to get some basic information on the pbLua File System, such as the total number of file descriptors, the total number of FLASH blocks available for use in the file system, and the size of each FLASH block.
-- Return information on the pbLua filesystem function FileSysInfo() files, blocks, blockSize = nxt.FileSysInfo() print( "Total File Descriptors -> " .. files ) print( "Total FLASH Blocks -> " .. blocks ) print( "FLASH Block Size -> " .. blockSize .. " bytes" ) end
The results look something like this:
> FileSysInfo() Total File Descriptors -> 31 Total FLASH Blocks -> 398 FLASH Block Size -> 256 bytes
If you're an experienced programmer, then you've probably come across the term "file handle". For those of you that have not, a file handle is essentially a unique value that you pass to the function calls in the file system that refers to a file. Normally you get that handle when you create or open a file.
File handles are normally arbitrary numbers that are assigned by the operating system. In the pbLua file system, the file handle is actually the number of the file descriptor in the file descriptor block.
we can take advantage of this to write a little function that dumps the detailed information for any descriptor. We'll write this function so that it can dump any number of continuous descriptors:
-- Return information on the pbLua filesystem
function DumpFileDesc(from,to)
print( "Type Name Block MaxBytes CurBytes CurPtr" )
for h=from,to do
print( string.format( "%4i %12s %5i %5i %5i %8i", nxt.FileHandleInfo(h) ) )
end
end
You might want to refer to the pbLua File API document for details on the return values from the nxt.FileHandleInfo() function. Of particular interest will be the file descriptor type...
The results of running the test program look something like this:
> DumpFileDesc(0,31) Type Name Block MaxBytes CurBytes CurPtr 3 pbLuaFileSys 0 512 0 1208832 1 ÿÿÿÿÿÿÿÿÿÿÿÿ 65535 65535 0 17985792 1 ÿÿÿÿÿÿÿÿÿÿÿÿ 65535 65535 0 17985792
You might be asking yourself what all those "ÿÿÿÿÿÿÿÿÿÿÿÿ" characters are doing in the file names? They are actually the 0xFF characters that make up an erased file name.
In fact, that's what all the values are in a free header ready for a new file (Type 1).
So let's move along and create a new file and see how the file descriptors change...
We'll create a simple file to test the operation of the pbLua File System. Files can hold all kinds of information. You might have text that you'll compile into a program, or you can have raw data that you've logged and want to save for later.
Let's start by deciding what text to put into the file, and in keeping with traditional programming practice we'll make it a function that will print the familiar "Hello, World" greeting on the NXT LCD, like this:
function Hello() nxt.DisplayClear() nxt.DisplayText( "Hello, World!" ) end
Now, programmers familiar with Lua will know that the function Hello() can be dumped as a string that represents the compiled bytecode of the function. As a matter of interest, the original text is 54 bytes long, while the resulting compiled code is 234 bytes.
So the question is how to represent the Hello() function as a string that we can save. Sure, we can build it up a bit at a time using separate strings, but it gets really confusing deciding which quote marks to use when we have strings inside strings.
Fortunately, the Lua language has a special kind of string that is useful for strings that span multiple likes, and they use the "[[" and "]]" characters as delimiters, like this:
programString = [[ nxt.DisplayClear() nxt.DisplayText( "Hello, World!" ) ]]
At this point, programString is just that, a string that has the text that makes up the program. Before we store it in a file, we'll need to know how long it is, and it would be a good idea to make sure it works too!
The following example first prints out the length of the programString variable, and then shows how to compile a string into a function that you can run to test how it works:
-- Print the number of bytes in the string: =string.len( programString ) -- Or do it a bit fancier: print( "programString has " .. string.len( programString ) .. " bytes" ) -- Compile the string into a function: test = loadstring( programString ) -- And run it, but clear the display first: nxt.DisplayClear() test()
So now that we understand a bit about how strings can be created and how they can be compiled and executed, we can move along to actually creating a file. And don't worry, in a few minutes we'll learn how to compile or run files without reading them into strings first.
I mentioned that the pbLua File System has some quirks that you'll need to work with and creating a file is one of them. Because the FLASH blocks allocated to a file must all be contiguous (one after the other) we have to give pbLua a clue and let it know how many bytes we're going to store.
Now, it's not as bad as it seems. If you ask for more bytes than you need, the file is truncated to the right length when you finish writing. The leftover space is still available. There's also a handy default value of 1K (1024 bytes) for the size, so as long as you're writing less than that you don't even need to supply the number of bytes.
Here's a complete example of creating a file, wrting the contents, and then closing the file handle. Note how the handle returned by the create operation is used to thell pbLua where to write the string, and which file to close:
programString = [[ nxt.DisplayClear() nxt.DisplayText( "Hello, World!" ) ]] -- Create the file, save the handle so we can use it later... h = nxt.FileCreate( "hello", 128 ) -- Now write the string... nxt.FileWrite( h, programString ) -- And close the file... nxt.FileClose( h )
If you're like me, the first thing you'll do is have a look at the file descriptors to see what has changed:
-- Now let's dump the first few file descriptors to see what we get: > DumpFileDesc(0,3) Type Name Block MaxBytes CurBytes CurPtr 3 pbLuaFileSys 0 512 0 1208832 5 hello 2 54 0 1209344 1 ÿÿÿÿÿÿÿÿÿÿÿÿ 65535 65535 0 17985792 1 ÿÿÿÿÿÿÿÿÿÿÿÿ 65535 65535 0 17985792
Cool! The first descriptor is in fact the hello file, it's a file header that can used for reading (Type 5), it starts at logical block number 2, and it's got 54 bytes in it.
Once a file has been written and closed, you cannot change any data in the middle, or add data to the end. That's just part of the challenge of a basic FLASH file system.
Now let's see what kinds of things we can do with the data in the file.
Reading a file is a bit simpler than writing it, but it has many similarities. The first thing you do is call nxt.FileOpen() and pass in the exact filename that you want to read from. You'll get a file handle that you can read the data from.
Now you have a couple of options. You can read the whole file and save the result in a string, or you can read the file a line at a time, or you can read any number of bytes from the file at once. Read the deatailed description of the FileRead() in the pbLua File API document.
The following examples show a number of different ways to read the hello file, along with the results:
-- Read the whole file as a string
h = nxt.FileOpen("hello")
s = nxt.FileRead(h,"*a")
nxt.FileClose(h)
print( s )
nxt.DisplayClear() nxt.DisplayText( "Hello, World!" )
-- Read the whole file a line at a time
h = nxt.FileOpen("hello")
repeat
s,n = nxt.FileRead(h,"*l")
print( n, s )
until nil == s
nxt.FileClose(h)
-- Note that the value returned for the last line is nil, we've -- arranged to have it printed here. Normally you would not print it. 18 nxt.DisplayClear() 34 nxt.DisplayText( "Hello, World!" ) 0 nil:0x0
-- Read the whole file 7 bytes at a time
h = nxt.FileOpen("hello")
repeat
s,n = nxt.FileRead(h,7)
print( n, s )
until nil == s
nxt.FileClose(h)
-- Note that the 3rd and 4th line are actually one set of 7 bytes from the file -- The embedded line end causes the line to be split into two lines when displayed! 7 nxt.Dis 7 playCle 7 ar() nx 7 t.Displ 7 ayText( 7 "Hello 7 , World 5 !" ) 0 nil:0x0
I'm sure that you are now thinking of all kinds of ways to make this work for you, so let's see about the other things we can do with a file besides writing and reading them.
In the previous section, we learned about reading files all at once or in little pieces. That will be useful when we get around to more powerful concepts like datalogs. In the meantime, we'll want to figure out how to compile and run files without having to read them into a string and processing them with the loadstring() function.
Fortunately, standard Lua systems have two functions that really help out here, and we've added then to the NXT API:
| Function | Description |
| nxt.loadfile(name) | Compiles the text from the file name and returns the results as a function that can be executed later |
| nxt.dofile(name) | Compiles the text from the file name and immediately executes it |
This makes it very easy to do some of the things that we want to do with files on the NXT, one of which is run them. But be aware, make sure you test your programs before committing then to FLASH. You don't want to be erasing files all the time when making changes. But don't worry too much - in normal use, you'll likely never hit the FLASH sector lifetime limit.
Here are some examples of how to use the nxt.loadfile() and nxt.dofile() functions. The first one reads the file, compiles it into a function, and then runs the function.
-- Read a file into a function that can be executed
f = nxt.loadfile("hello")
-- And now run the funtion to see the result
f()
This example does it all at once, loading and running the file immediately without the need for an intermediate variable.
-- Read the file and execute it all at once
nxt.dofile("hello")
Wait a minute, I can hear it now. Wouldn't it be great if you could choose a file to run from a menu on the LCD of the NXT, and it would be neat if you could return a number of different things, like the filename, or the test in the file, or even compiling or running the file.
pbLua can do that too, with the nxt.FileChooser() function, which is completely documented in the pbLua File API You can scroll through all of the files using the left and right arrow buttons on the NXT, and you select the file using the ornage button. If you don't want to select a file, then press the grey rectangular button.
The default, safe action of the file chooser is to just return the name of the file. Here's an example of how it works:
-- Read the file and execute it all at once
nxt.dofile("hello")
And an example that prints the entire text of the file that was chosen:
-- Run the file chooser and return the name of the file chosen:
print( nxt.FileChooser("name") )
-- Or more simply...
print( nxt.FileChooser() )
--codeExampleEnd 13
--codeExampleStart 13 -----------------------------------------------------------
-- Run the file chooser and return the contents of the file chosen:
print( nxt.FileChooser("file") )
And an example that loads the file into function:
-- Run the file chooser and load the contents of the file into a function:
=nxt.FileChooser("loadfile")
-- Whoops, let's put it into a funciton and execute it...
f=nxt.FileChooser("loadfile")
f()
And an example that executes the file immediately:
-- Run the file chooser and execute the contents of the file
> =nxt.FileChooser("dofile")
And a final example that is the same as the previous one but reduces the default timeout to only 5 seconds from 15. The timeout is exented whenever you press one of the grey arrow buttons on the NXT.
-- Run the file chooser and execute the contents of the file, but only give the
-- user 5 seconds to make up their mind
> =nxt.FileChooser("dofile",5)
If you've read this far, congratulations! There's one more thing that you will want to have, especially if you're planning to keep certain programs in your NXT. You won't always want to use a console to run your programs, so is there way to run a file even if you have no console?
Of course, the answer is yes!. And the key is in understanding the pbLua startup sequence. Here's how it works:
So that's it? All we have to do is make a file called "pbLuaStartup" and it will run automatically? Yep - so let's do it, and make it run the FileChooser() and the play a little song so you can see the display. You can probably see where we're going with this:
programString = [[
nxt.FileChooser("dofile")
melody = string.char(0x02, 0x00, 0x04, 0x00,
0x01, 0x00, 0x04, 0x00,
0x02, 0x00, 0x02, 0x00,
0x03, 0x00, 0x02, 0x00,
0x01, 0x00, 0x01, 0x00,
0x03, 0x00, 0x01, 0x00)
while 0 == nxt.SoundMelody( melody, 1 ) do
-- do nothing
end
]]
h = nxt.FileCreate( "pbLuaStartup", 512 )
-- Now write the string...
nxt.FileWrite( h, programString )
-- And close the file...
nxt.FileClose( h )
Now, unplug the USB cable if it's connected and turn your NXT off by pressing the grey rectangular button for 2 or more seconds. Then turn on the NXT. You should see it figure out that there's no USB cable, then it waits for 10 seconds for the user to chose a console. If none is chosen, then you'll get the FileChooser() running, and if you choose the hello program it will display "Hello, World!" and play a liitle tune for a few seconds. Then the whole cycle starts again at the console chooser.
The NXT File API is very powerful, and we've gone over pretty much everything we'll need. Once again, refer to the pbLua File API page to get more detailed information on each function.
There will be another tutorial that uses the File API to implement datalogs, which is an exciting way to use the NXT to log data in real time and then process it later.
by Roberto Ierusalimschy
by Jon Bentley