HOME  |  NEWS  |  BLOGS  |  MESSAGES  |  FEATURES  |  VIDEOS  |  WEBINARS  |  INDUSTRIES  |  FOCUS ON FUNDAMENTALS
  |  REGISTER  |  LOGIN  |  HELP
Comments
You must login to participate in this chat. Please login.

Catching up on archive lecture.

Gold

still getting caught up, better late than never.

Iron

hello all from Edmonton, Alberta

Iron

Thanks for archiving these sessions, well worth the review

Good presentation as an introduction and overview.  It sounds like you must have a minor in Computer Science in addition to your EE training.  You touched on, hinted at, or alluded to a lot of concepts.  I would like to see/hear more on fragmentation, overhead (space overhead, e.g. meta-data and block size allocations ... and time allocation in terms of search efficiencies).  Looking at the week's syllabus I am hoping you will get to these topics / details ... I would also like to hear more on where / how these file systems are used, which was indicated sometimes specifically, sometimes more generally ...

thanks Eric and Lauren
Iron
I'm personally used "raw" FAT system realisation at 1987 for antivirus prototype
Iron

I mostly work on projects with all flavours of ARM. I've worked with FAT and ext filesystems in embedded systems

Iron

very good,

see you all tomorrow

Iron

Thanks...I'll try to be earlier tomorrow.

 

Iron

can you talk about the google's distributed file system? Are they very different than the traditional file system or just another subset from your presentation?

Iron

Great introduction material. Thank you.

Iron

Thank you for nice presentatin. See you tomorrow.

Iron

Working with FAT12/16/32 want to know more about JJFS, ext3 and ext4

Iron

Thanks everyone, it's been a pleasure, see you tomorrow for the lecture on FAT's internal architecture and details about its operation.

Blogger

jnettleton: Will you be discussing fault tolerance (because of power cycle, exceptions, bad clusters, etc)?

Yes, I will be discussing fault tolerance in the context of the FAT file system on Wednesday (day 3).
Blogger

Mark1200: have you used a file system with your NAND storage or did you use it to store plain raw data?

Blogger

Atlant: I'll be there all week.

Blogger

Thanks, looking forward to the folowwing sessions.

Thanks, Eric. The video HD application would be security, so the camera would be enabled by some sensor net, then stop recording when the sensors return to normal or the media fills up. After a block is transferred to the second storage location and successful transmission is verified, a n erase operation would be performed on the original data..

Thanks, Eric!  This has been excellent.

Iron

eric: Thanks -- See you on Friday! (Well, whichever days of the week that you're here.) And "Thanks!"

Iron

Atlant: Unfortenately, I won't be covering NAND-specific file systems (nor know much about UBIfs) , but I'll cover FTLs which would allow you to use FAT.

Blogger

eric: I'm sorry but I don't remember. If I can, I'll check that log before Friday.

Iron

I have worked with FAT12/16/32 our own implementation.  I have also written custom file storage systems for RAM, NOR, NAND and MNAND storage.

Iron

"oven" -> "often"

Iron

Atlant: were they write errors as reported by the device's status register or was it read errors after the page had been written? 

Blogger

eric: Even now, we've found that UBIfs has certain pathological error conditions from which it doesn't recover gracefully. It tends to try to immediately re-use the just-failed block and because you're trying to write exactly the same data to exactly the same block, it oven fails again in exactly the same fashion. In my favorite logfile, this went on for *SEVERAL HOURS*!

P.S.: I'd love to hear about good alternatives to UBIfs! ;-)

Iron

Johan Hartman: For wear and flash media, most FTLs (Flash Translation Layers) will implement some sort of wear leveliing, so that you don't have to worry that much about excessive wear. I will detail that in lecture 5 (Friday)

Blogger

eric: ("but write errors are really uncommon") You never met the MLC Flash that we were using for a while! ;-)

Iron

So what then is the lesser of these evile's. Living with having to recover from open files at every boot-up - if that is possible.  Living with excessive wear because of many re-open, append data and close.  Or is there some other clever mechanism that you will be addressing during the course.

Atlant: right, maybe not 100% deterministic, but write errors are really uncommon, if you don't try to reuse failed blocks. 

Blogger

Johan Hartman: Pre-allocating a file (seeking until desired length can do that) allows you to prevent some of the metadata updates. Some FS implementations also have metadata caching features that will cause no metadata to be updated until the file is closed or the metadata is explicitely flushed

Blogger

eric: As long as one accepts that the ratio can be quite a few orders of magnitude. And I think it's only deterministic if one can assume a "perfect" Flash device; the moment you start to encounter read- or write-disturb errors, all assumptions of determinism fall apart.

Iron

Atlant: the worst sector write time can be deterministic, althought not as good as the average sector write time.

Blogger

Johan: That sort of approach, though, sounds like it would require you to write your own custom Flash File System. The widely-available FFSs try to emulate ordinary magnetic disk drives and they often end up being too-clever for their own good.

Iron

Alaskaman66: a volume on a separate, dedicated device might be the best, if you're also doing some other operations on other files in your application.

Blogger

Johan: In environments with "big" operating system (such as Linux), you also see effects from the system "caching" your data. Linux may be able to "write" (store in its main-memory file cache) a few dozen megabytes but the next byte you write may force a real file-device operation; that write may go *VERY* slowly!

Iron

Can you mitigate these problems by allocating and writing a large file - then only overwriting blocks in it as you log data.  If you choose teh erase state data of the flash for the default data you write to it, then theoretically, you dont have to erase blacks to write new data?

make that "...one more page..."

Iron

Johan:

 

Time on a Flash device usually *ISN'T* deterministic because you may be able to quickly write on more page to an already-erased Flash Physical Erase Block or you may not have any more un-erased pages and you now need to garnage-collect the free space and erase one or more full Physical Erase Blocks.

Iron

Johan Hartmann: yes the metadata has to be written everytime you grow the file, or even write to the file, since you update the "last write" date/time. That would mean greater wear on those sectors that store the file metadata, but on Flash Memory, in order to use a traditional file system like FAT, you need some translation layer. Those often implement wear leveling.

Blogger

@Atlant.  That means you have a difficult choice to make between wearing the flash media and having an open file corrupted by a power failure.  Is the time to append a block of data to a file on a flash media deterministic?

 

erice (re recovery schemes): Yes, that's the sort of thing I was referring to. And even if they do attempt to implement recovery, users expect the embedded system to respond immediately upon power on and not spend half an hour running fsck. ;-)

 

Iron

RMRSS: the file size is limited because the metadata associated to a file has a 4 byte field for the size in bytes (max 4GB). It cannot be curcumvented, unfortunately.

Blogger

I have an application question that considers speed differences. Say you are using an HD camera, and a DSP to convert the video to MPEG-4 on the fly. You need to send this video to an external location over a bandwidth limited link. Obviously, the file system (say NTFS) that stores the MPEG files needs to keep up, but the transmission can be done slower and later. Would you use a seperate file system and storage to transfer MPEG files to a "comm" file for transmission. Maybe a second physical system all by itself associated with an RF link? Thanks

...Many Flash File Systems also don't keep "last accessed" dates so that at least *THAT* metadata doesn't need to be updated.

 

Iron

Johan: Yes, every time you modify a file on the Flash file system, the metadata needs to be modified. The flash file systems try to be "clever" about not ,ocating the metadata in a fixed spot on the Flash chip so they don't wear that spot out early. ...

Iron

Thanks Eric and Lauren; Excellent!  

Iron

@Atlant: Also, embedded systems don't usually implement complex recovery algorithms as they're limited resource-wize.

Blogger

eric: And today, more and more PCs are laptops with batteries so mains power failures are becoming less of an issue for many (but not all) PCs.

Iron

what is the mechanism behind the 4GB file limit ie is it the same for FAT16 as FAT32  and can it be circumvented and if so is it practical?

Iron

Thank you. Very good presentation

Iron

What is the impact on a flash based system if you close a file, then re-open it often to append to it.  I would guess that the file meta data have to be re-written evry time?

@cghaba: Mostly, yes. It's worsened by the fact that some devices might be powered off by the user at any time. Of course, PCs might use more caching and file buffering which might make matters worse, but since they're plugged into the mains, that's less of an issue.

Blogger

Great presentation, good information

Iron

kevenm1, et al.: Embedded systems often have fewer resource from which to recover minor corruption in their file systems than do full-scale PCs. Thus minor corruption can lead to a "bricked" embedded device.

Iron

thanks for the presentation. pretty informative/

Iron

Reset on embedded system can be as disruptive as on a PC, and in many cases worse.

Iron

@jareestad: They are typically more robust when the file is closed, since it's metadata will not be overwritten, which might be a source of corruption. I will go in more details about how corruption can occur in day 3 lecture.

Blogger

Thanks!Good Presentation.

 

Hi Eric,

Definitely for my purposes I would liek to learn more about the file systems used with Power-PC microprocessors.

Thanks for the excellent presentation!

Iron

Thanks Eric , its good info

 

Is the effect of a reset on an embedde FS the same as for a PC system?

Iron

very useful information for me, Eric, excellent job too..

 

Iron

Very informative, thank you

 

Thanks, Eric. Very informative, and a subject most folks don't want to deal with

 

good session today

Iron

I will be storing simple text data in 7 bit format, 

The data source will be from scanned optical sources.

 

Iron

jaarestad: Robustness against power-cuts can be a *HUGE* issue with embedded filesystems.

Iron

Thanks til tomorow.

 

Iron

Thanks Eric, great presentation.

Iron

Thanks Eric.  These will be very useful lectures!

 

Thank you for today's lecture Eric

Iron

@Atlant - interfacing with "real world" (reading files created by other devices/users) may be the only reason...

Iron

thanks, see you tomorrow

Certainly could, depends on application

Iron

Good presentation

 

Iron

Not if by "short file names" you mean 8.3 filename. Anything else is considered "long".

Iron
What is the impact of file-open and file-close atomicity? Are some file systems more robust to corruption if an error occurs when a file is "open"?
Iron

thanks for the overview Eric

Iron

@david.rea: something like ".../logs/YYYYMMDD/HHMMSS.txt" does not work?

Iron

Any other UBIfs users here?

Iron

caa028: Because the device might interface to other devices "in the real world"?

Because you're encoding (e.g.) a date/timetamp in the filenames of logged data?

Iron

> Why would one need long file names support on an embedded device?

Save log files in YYYYMMDD-HHMMSS.txt format?

Iron

Depends on your application

Why would one need long file names support on an embedded device?

Iron

No zfs users here?

Iron

(If I said F11ODS-1 and F11ODS-2, would anyone know what I'm talking about? ;-) )

Iron

Will you be discussing fault tolerance (because of power cycle, exceptions, bad clusters, etc)?

Iron

being a Linux guy I lean towards Ext2/3/4

If it is like computers probably NTFS but I think that FAT32 would be easier to learn at first.

 

Iron

FAT32 also has a large overhead for small files on large devices.

Iron

Ext2/3/4, HFS/HFS +, 

I have used FAT 32 but have not done any Programming.

Iron

Haven't yet worked with a file system in an embedded capacity. Would probably be working with a FAT variant since we're looking at eventually implementing embedded Linux.

Iron

ext2/3 in particular but i am interested by the other one

I only heard about FAT and ext2/3/4. Don't know the ohters. I think I would like to work with FAT32

Iron

Interested in FAT and FAT32 which seem to be in everything

An embedded system that can survice loss of power during updates is handy.

I would like to use FAT

Iron

FAT16 and FAT32 is what we are using.

The FAT man cometh

 

Iron

Our only expirence is with FAT.

Iron

Ext3, FAT32, UBIfs

Iron

@Atlant - as log as I don't have to mop the floor...

Iron

I think B-Tree stood for binary Tree in 1974

 

Iron

caa028: You'll have to stay late and clean the erasers!

Iron

A bit late for the live session today...

Iron

Not used embedded file systems yet.

Iron

No experience on Programming.

Iron

problem with slides loading

Iron

I have not used a file system in any designs yet.

I have used mostly 8 bit and 16 bit micros, but I would be receiptive to more advanced micros for right application.

Iron

PIC, Atmel and Cortex

Ankura: Using INternet Explorer, the Audio Player is appearing several paragraphs *ABOVE* the box I'm using to type this chat comment. It's just "one line" high.

Iron

(ankura - it's right below the title of the page; try reloading or re-logging in if you can't see it)

Iron


@ankura  keep refreshing, use firefox or Google Chrome

Iron
Iron

MAKE SURE YOU HAVE LATEST PLUGINS

 

Iron

ATMEL ATmega 8bit

Iron

PIC, MegaAVR, Cortex

Iron

Depends on the client requirements, but have used MSP430, PSoC, some of the smaller Atmels.

Iron

used both types microcontrollers (freescale 9s12) and microprocessors (atmel SAM9263)

 

Iron

I can't see the player, what's wrong ? Can someone help me please?

Iron

TI CC111x 32K RAM

Iron

No experience to this.

 

Iron

8 bit AVR and AVR Xmega

Atmel 8/32  bit, TI Concerto, Microchip 8/32 bit.  Using them all.

Iron

ARM based. Just wanted to learn more.

 

small processors, currently Coldfire V2 MCF52255

Arm cortex, Renesas
Wieslaw

Iron

I am using Coretex a9

Iron

ARM and 8051, legacy 80188.

Renesas r8c, MSP430, Pic,

Iron

ARM for mobile phones.

Iron

~400 MHz PowerPC.

Iron
No FS experience in FS, except for Microchip FS in a web demo application. Interested in learning more.
Iron

Never wrote anything for a FS, but use FAT16/32 and NTFS in computer systems

Iron

try refreshing page

 

Iron

No work with FS for embedded systems

I have no audio, just noise

never used a file system in embedded system

Iron

Yes we have an open source one working on an embedded platform

Worked with fat16/32 FS 

I've used a file stack in a supplied framework, never understood it...LOL.

Iron

used FAT system

 

Iron

Never used in embedded system but expect to in near future. Great timing for this class!

Iron

Never used file system in embedded application

Iron

(Ankura: Press the "Play" arrow on the audio player.)

Iron

have used a fat based fs

Iron

Worked extensively with various file systems. Does it ever go well?

Iron

Have not before but working on a project using it now.

 

yes, just finished writing a fat16/32 file system for an embedded application accessing SD cards

 

Iron

yes a use my own file system

I have not used a file system in an embedded system

No real experience, educate me.

Not yet for me... new to me...

 

Iron

I have not used FS stack and not implemented a FS.

Iron

Have implemented FAT for RAM PCMCIA card removeable media.

Slide 5 -- QDOS or Seattle DOS? in 1981?

 

Iron

Never used a file system stack

Iron

try refresh a few times and use firefox

Iron

Using Linux + UbiFS.

Iron

I don't hear anything, plus the player dis not launch. Whats wrong?

Iron

good afternoon, everyone

Iron

Good morning from CA

Iron

Hello, from Moncton NB

hello from Calgary!!

 

audio up and clear

Iron

here we go audio is up!!

Iron

looks like a great class, cold NJ

Blogger

(This comment intentionally left blank)

Iron

Sacramento CA checking in.

Iron

Good day from sunny Phoenix

Iron

When will it start?

Iron

University of NE in

 

Iron

Green Bay WI here

 

Iron

hello from Canada

 

Iron

Hello from Greensboro, NC

Iron

Hello from Edmonton, AB

Iron

hello! It's a nice day! from Richmond, BC

Iron

Hello from Eugene!

 

Hello from Albuquerque.

Iron

hello from Toronto

 

Iron

helo from silcon valley west

 

Iron

Hello form Toronto.

Iron

@cghaba: buna, in sfarsit mai vad pe cineva din Romania ;)

Iron

hello from Timisoara, Romania

Iron

Hello, from (sunny!) Pittsburgh, PA

Iron

Hello from Minnapolis.  26 degF today, and snowing.

Iron

Good evening from Iasi, Romania

Iron

Less than 1 hr to-go....

Iron

Good afternoon from Milwaukee!

Iron

Afternoon from St. Louis

Iron

Greetings from Raleigh NC

Iron

Yep, I'll take this weather.

Hi from Texas too.  It sure is warm for March. ~73degF at noon.

Iron

Please join our Digi-Key Continuing Education Center LinkedIn Group at http://linkd.in/yoNGeY

Platinum

Hello from Reading, UK

Iron

Saludos desde/Greetings from  Ensenada Baja California

Iron
Good morning from Huntington Beach, CA.
Iron

Good Morning from Vancouver

 

Iron

Good Morning from Farmington NM

 

 

Good Morning from Mobile, AL

Greetings from Portland Oregon

Iron

Greeting from Herndon, VA

Iron

Hellow everyone, good morning from Vancouver Canada.

 

Iron

Hello from Oregon, pdx

Greetings from eastern Mass. A sunny and cool 37°F.

Iron

Good Morning from San Jose, CA

Cloudy: Low 42°F High 60°F.

 

Iron

Greetings from Chicago.

Iron

@Lauren_Muskett   It would be nice if you folks can start the audio with 2 minutes of so of music, so we can get our audio players "engaged". I usually have to restart mine at least once, and miss your introduction and the first words of the presenter.

Iron

The streaming audio player will appear on this web page when the show starts at 2pm eastern today. Note however that some companies block live audio streams. If when the show starts you don't hear any audio, try refreshing your browser.

Platinum

Greetings from Vermont, 27 deg F and overcast

Iron

Good morning from sunny NY.

Iron

Be sure to click 'Today's Slide Deck' under Special Educational Materials above right to download the PowerPoint for today's session.

Platinum

How's the weather in Scottsdale, MB ?

Good morning from Scottsdale, AZ

Iron

Good Morning from Sunny Boston!  Slides 1-25 ,Check 

Iron

Good morning from Sunny SE Lake SImcoe Ontario in Canda.

Iron

Good Morning from GA

Iron

@Bartholemew

Multiple type of file systems are routinely done on large systems. Our Linux servers use one style for the boot and another for the data -- for journaling. The boot volume does not change much. It's done routinely and much more than most people realize. On MCUs with small file systmes -- mostly SD cards etc.  I would guess not so much. A suite of hard drives changes the picture.

Iron

Good morning from VT.

Hello from snowy Valdez once again.

Reviewed day 1 slides; any comments on/examples of systems where more than one FS is used, to balance different needs in different parts of the system?  e.g. high reliability for mission critical firmware in one part of the system but more lightweight storage in another part of the system, for user data that is expected to be backed up?

Slides look super.

see you soon

Iron

Beware the ides of march !



Partner Zone
Latest Analysis
Take a look at the top 20 US undergraduate engineering programs. Then tell us -- did your school make the cut?
Producing high-quality end-production metal parts with additive manufacturing for applications like aerospace and medical requires very tightly controlled processes and materials. New standards and guidelines for machines and processes, materials, and printed parts are underway from bodies such as ASTM International.
Engineers at the University of San Diego’s Jacobs School of Engineering have designed biobatteries on commercial tattoo paper, with an anode and cathode screen-printed on and modified to harvest energy from lactate in a person’s sweat.
A Silicon Valley company has made the biggest splash yet in the high-performance end of the electric car market, announcing an EV that zips from 0 to 60 mph in 3.4 seconds and costs $529,000.
The biggest robot swarm to date is made of 1,000 Kilobots, which can follow simple rules to autonomously assemble into predetermined shapes. Hardware and software are open-source.
More:Blogs|News
Design News Webinar Series
7/23/2014 11:00 a.m. California / 2:00 p.m. New York
7/17/2014 11:00 a.m. California / 2:00 p.m. New York
6/25/2014 11:00 a.m. California / 2:00 p.m. New York
5/13/2014 10:00 a.m. California / 1:00 p.m. New York / 6:00 p.m. London
Quick Poll
The Continuing Education Center offers engineers an entirely new way to get the education they need to formulate next-generation solutions.
Sep 8 - 12, Get Ready for the New Internet: IPv6
SEMESTERS: 1  |  2  |  3  |  4  |  5  |  6


Focus on Fundamentals consists of 45-minute on-line classes that cover a host of technologies. You learn without leaving the comfort of your desk. All classes are taught by subject-matter experts and all are archived. So if you can't attend live, attend at your convenience.
Next Class: September 30 - October 2
Sponsored by Altera
Learn More   |   Login   |   Archived Classes
Twitter Feed
Design News Twitter Feed
Like Us on Facebook

Sponsored Content

Technology Marketplace

Copyright © 2014 UBM Canon, A UBM company, All rights reserved. Privacy Policy | Terms of Service