Latest post Wed, Jan 30 2019 11:16 AM by derDoyle. 23 replies.
Page 1 of 2 (24 items) 1 2 Next >
Sort Posts: Previous Next
  • Tue, Feb 5 2013 1:39 AM

    • Robert B
    • Not Ranked
    • Joined on Sun, Aug 7 2011
    • Los Angeles
    • Posts 83
    • Points 1,005

    Anyone Fooling Around with Sparse Disc Bundles?

    Adventures in Sparse Disk Bundles

    After reading an article about using OSX's Sparse Disc Bundles to help media manage FCP X, I thought I would give it whirl in Media Composer. 

    Let me preface this by saying our pieces are normally in the 5-10 minute arena so the cumulative size of AMA sources, Avid MXF Media and Project Assets rarely exceeds 1 TB. 

    The Apple Sparse Disk Bundle is created through Disk Utility and is used by programs such as Time Machine for backups. Unlike a regular disk image, Sparse Disk Images expand as stuff is added to them whereas a Disk Image pre-allocates the space. The "Bundle" option saves the disk image in small 8MB "bands" - this allows you to recover a Bundle if a band becomes corrupted - at least in theory. 

    I've now used this workflow on 5 projects and haven't had a problem yet - HOWEVER - this is experimental, PROCEED AT YOUR OWN RISK. 

    I'm posting this so that other brave souls can play with this and report back. Please share your joys and nightmares. 

     

    The Benefits

    1. When you double-click on a Sparse Disk Bundle, it mounts like a virtual harddrive. When working in the Avid, you can specify your Media Creation settings to write to the virtual harddrive. This is great for keeping Avid MediaFiles separated by Project - the bane of my existence as I routinely switch between 4-5 projects a day. Its like a folder with superpowers. It can mounted AND unmounted - thus hiding it from the prying eyes of Media Composer. 

    2. As long as the SDB isn't already mounted, double-clicking on the file over your Gigabit network or Fiber network mounts the volume on the remote machine AS IF it were a local disk. Imagine this scenario: You finish your edit on your main system, close out of Avid, UnMount the volume (eject it), Mount the SBD on your Secondary Computer (IMAC, MAC MINI) over the network, open the avid project and perform the export on the secondary system. This allows you to jump to the next project on the main system and keep working while the tedium is handled by the second machine. This also works for transcoding operations. You need two Avid licenses of course.

     NOTE: You can't work on the same project with two systems simulaneously

    3. Sick of relinking AMA sources when transferring to a new drive?

    When handing off a project to another editor, normally the editor must point the AMA master clip the to the new drive/file path before he/she can continue working. When using SDB's, you copy the bundle file to the new harddrive. When the second editor mounts the volume, the drive name is the same and all AMA sources contained within in it stay linked. 

    4. Archival made easy. Archiving Projects is as simple as dragging the bundle file to your backup drive or tape drive. One file to rule them all. 

     

    The Cons

    1. It feels kind of scary putting all your eggs in one file basket even though its supposed to have some corruption protection via the "bundles". 

    2. 80% sure, Sparse Disc Bundles are a Mac-only format that works with Lion, Snow Leopard and Mountain Lion. 

    3. You must specify a maximum volume size when creating the SDB; moreover, the drive you are creating the bundle on must have more free space than your maximum volume size to create it - BUT you can move the bundle to a drive with less available space as long as the bundle's expanded size can fit.

    NOTE: I'm have no idea what the tradeoff is to making your bundle maximum size HUGE vs. keeping it reasonable.  

    4. Not sure if data integrity is maintained when these are copied to other file systems and back. 

    5. The SDB will automatically expand as more files are added but it won't contract when files are deleted. Luckily for me, my projects never get smaller:)

     

    The Scary

    1. Volumes can be resized through Disk Utility but its a little unnerving. 

        a. You select the SBD in Disk Utility and use the "Resize Image" button. You choose the "resize volume + partition" option.

        b. Type the new size and hope the computer doesn't freeze. You will now notice that nothing seems to change. 

        c. Due to a disk utility bug, it will resize the volume but will not resize the partition as it should.

    The workaround -  You must now select the volume and manually resize the partition to your new desired size. I've found you have to make slightly smaller than the volume or you get an error.  Resizing the partition does not erase the data held within - but it definitely kind of freaks you out as its processing.

    There is also a terminal command that can make the volume larger. 

    2. Volumes CAN be contracted if you've deleted a bunch of files. This is done by a terminal command. I will most likely just         leave them bloated vs. taking the risk.

    If this were a mission critical project, I would make a copy of the SBD before trying anything like a resize or contract.

    A LESS SCARY METHOD FOR RESIZING - but longer

    1. Create a new SDB that is the new size you want, mount the new volume and the old volume - now just copy from the old to the new. Feels way safer, but will take a lot longer. 

     

    That's all I've got for now:)

    -Robert

     

     

     

    OFFICE: 3.5 GHz Core i7 Late 2013 Imac running OSX 10.11 with 32GB RAM. NVIDIA GTX 780M 4096 MB. Avid Media Composer 8.5/ Pro Tools 12.4 MOBILE: 17"... [view my complete system specs]
  • Mon, Feb 18 2013 4:12 PM In reply to

    • Zak Ray
    • Not Ranked
    • Joined on Fri, Jan 28 2011
    • Posts 22
    • Points 260

    Re: Anyone Fooling Around with Sparse Disc Bundles?

    We did the same thing after hearing about the FCPX method, tried it for about 6 months, and went back to the old way of doing things. It was admittedly a nice way to work, but the cons (the disk resizing issues, concern over long-term archive accessibility, and for us in particular, the difficulty of restoring a file from a backup when needed) outweighed the pros. Ah well, back to manual media management...

  • Wed, Jan 14 2015 7:04 AM In reply to

    Re: Anyone Fooling Around with Sparse Disc Bundles?

    Hey I know this post is a little older now but maybe someone else will stumble upon this and get my 2 cents as well.

    I'm preparing for some work coming up soon that's going to be avid based at a mac facility.

    I wanted to use the good old windows trick of mounting shared local folders as network volumes but Mac OS X doesn't allow this like windows does.

    I was noodling around for a long time to find something that worked as well and there really wasn't much. I considered sparsebundles but was concerned there might be unforseen consequences. Your post itself basically made me decide I wouldn't risk it at a facility and be responsible for the potential issues that could occur including data loss or inaccessability of an archive in future or any non-mac platforms or via obsolescence of the sparsebundle disk image. However I have figured out a way of combining the sparsebundle idea with another known workaroud for separating media at a folder level by project which involved symlinks. The methodology for the symlink workaround which I've adpated here can be found in this thread http://community.avid.com/forums/t/125833.aspx posted by user Kenton.VanNatten.

    Summarising Kenton's method for brevity, it's basically making a shortcut MXF subdirectory of an avid mediafiles folder on a drive root which links to an avid media files mxf folder located elsewhere probably buried in subdirectories. The key difference being that it's neither a 'shortcut' nor an 'alias' but a 'symlink' which seems to be roughly the same thing but an older method of doing the same sort of thing and achieved through terminal. (Don't worry, it's short and easy). Doing this with Mac OS X aliases doesn't work, but Mac OS X fully supports symlinks as they're a unix thing and Mac OS is built on Unix.

    So, Kenton's method allows you to redirect media avid generates and tries to send to it's root level 'Avid MediaFiles' folder to a subdirectory on the same drive but what was an issue for me was that you could still only have one Avid MediaFiles folder per volume which is an problem if you for example need to consolidate material from one project to another on the same volume (I know this is wasteful of space, but good for archiving since you have everything contained and just need to copy a folder). It's also a problem when people need to remotely access media over a network for an avid project whose media resides on directly attached storage connected to your machine, but you simultaneously need media from that same volume for a different project with different media that your locally accessing.

    Making sparsebundles solves this issue, but has all the drawbacks you mentioned, and symlinks only allows you to keep things neat when you're working one project at a time but still has a limitation of one Avid MediaFiles folder per volume. What I did was to create a sparsebundle image named after the project and I also appended 'symlink' to the name just to be meaningful and in that sparsebundle created an avid mediafiles folder within which resides an MXF directory that is symbolically linked to a subdirectory on the same physical volume. In this way, I can for example open a project which has media sitting in a folder named after the project but which accesses the media through a symbolic link in the root Avid MediaFiles folder on the drive and then consolidate media from that project to the Avid MediaFiles folder of a different project on the same hard drive in a different folder by mounting the sparsebundle for that project, selecting it as the target for the consolidation in Avid and then having that sparsebundle act as a conduit from Folder A on Drive XYZ to Folder B on Drive XYZ. The sparsebundle never grows any bigger because no media is ever added to it, it just funnels it through. This way you can keep multiple projects online by just mounting different sparsebundles but you have none of the associated drawbacks or risks of sparsebundles.

    It's important to note that I use sparsebundles as opposed to standard disk images because you must nominally have enough space for the media in your project on the disk image even if you don't use any of that space. You could do this with a traditional disk image, and select a large size, but that disk image actually consumes that quantity of space even if it is empty. A sparsebundle, can be set to any size you like, even one larger than the physcal volume on which it resides, but it will only ever consume as much space as the data you put in it. The only data you ever put in to this sparsebundle is a shortcut, so it never grows larger. My test image I made for this for example, is only 62KB in size, negligble, I was conducting the test with a project consisting of only one 200 MB media file and I set the maximum size of the sparsebundle to something many times that at 8GB, but even though it may theoretically reach a maximum size of 8GB, it should only ever be as big as 62KB. I don't see any reason why you couldn't set the maximum size to unrealistically huge sizes but I find the idea a little scary so I was conservative with my test. I'd probably not venture much beyond the size of the physical volume you put the bundle on to.

     

    So, step by step. (I'll simply say "create a symbolic link" rather than instruct on this since the above linked post explains that)

    1. On your physical storage volume create a folder and name it after your project Project A.

    2. In that folder create a subfolder called "Avid MediaFiles"

    3. Inside that folder create subfolder called "MXF"

    4. Go to disk utility, create a sparsebundle

    5. Set the maximum size to something at least large enough to house the estimated maximum size of all the media in your project, set the format to Mac OS Extended (journaled), select sparsebundle disk image as the Image Format and name after your project Project A and add 'symlink' to the name. Project A symlink Save as the same name.

    6. Mount your new sparsebundle

    7. Create a folder called "Avid MediaFiles" on the root directory of your sparsebundle

    8. create a symbolic link between the "MXF" subdirecory of the Project A folder on your physical storage volume and the "Avid MediaFiles" folder of your Project A symlink sparsebundle root directory. With the "MXF" directory as the source and the "Avid MediaFiles" folder as your target.

    9. In your sparsebundle, within the "Avid MediaFiles" directory, there will now be a folder called "MXF" and will have an alias icon (the little black curved arrow on top of the folder).

    10. As long as this sparsebundle is mounted, any media within the folder project A will be read by Avid as residing on the logical volume Project A Symlink even though, in reality, it's all on the same physical and logical volume as your original hard drive.

     

    I assume you can also share these sparsebundle (As mounted volumes) over a network and access them via AFP protocol as well, but I don't have any other computers on my home network to test this with and they're all connected via wifi anyway so it'd be useless for me. I was able to test whether or not Mac OS would at least allow me to share the volumes and it did so theoreticaly this works for this purpose. You can't however share the sparesbundle iteslf like a folder, you must mount it on your machine first, then share the logical volume, this means then that it if you unmount the volume, anyone else that is accessing it via your machine, will lose access to it. P.S. after I edited this post to fix up some typos and phrasing, all my list numbering was removed when I hit post. I can't get it back, so I've had to do it manually (although the list tags were still showing up in the HTML). Sorry for ugly formatting
  • Wed, Jan 14 2015 7:32 AM In reply to

    Re: Anyone Fooling Around with Sparse Disc Bundles?

    I've had to make this section a separate post because the forum keeps screwing up my formatting and removing all my linebreaks. For multiple projects, repeat the steps for each project. This allows you then to keep them all online simultaneously for as long as each sparsebundle is mounted. You can also consolidate media between projects this way if you want to double up. Again this is consumptive of space perhaps unnecessarily because you could just copy the clips between projects without copying the media but sometimes it's good just to keep everything contained. As an example, to do this, assuming you've followed the steps with 2 projects already, here's how you'd do it.

    1. Mount Project A symlink sparesbundle
    2. Mount Project B symlink sparsebundle
    3. Open project A in Avid
    4. Choose some media you would like consolidated from Project A in to Project B
    5. In the consolidation window, select Project B Symlink as your target drive and click 'consolidate'

     

     

    As far as Avid is concerned you've just copied media from drive Project A symlink in to drive Project B symlink but in reality you've just copied media from folder Project A to Project B.
  • Wed, Jan 14 2015 9:49 AM In reply to

    Re: Anyone Fooling Around with Sparse Disc Bundles?

    One third and hopefully final addition.

     

    I have succesfully tested whether or not I could mount the one of my sparesbundles over my local network, though I don't have Avid installed on any other machines and have only wifi network connectivity so can't fully test. I can say though that I was at least able to mount the sparsebundle stored on an external hard drive connected to my machine, via another machine over the network and still have the symbolic link remain intact. (with a caveat)

     

    It might seem obvious but it still tripped me up at first. You must remember to mount both the sparsebundle, and whatever volume your mxf media is actually stored on, otherwise the symoblic link is broken as it is referencing a path inside your systems /Volumes directory that doesn't exist because it isn't locally attached to your machine. If you mount both the physical storage volume and the sparsebundle, the symbolic link will be preserved on the client machine.

  • Wed, Jan 14 2015 6:39 PM In reply to

    • Isaac T.
    • Top 150 Contributor
    • Joined on Mon, Jan 9 2006
    • New York / Knoxville, TN
    • Posts 742
    • Points 8,765

    Re: Anyone Fooling Around with Sparse Disc Bundles?

    @AvowedMediaComposer: I read the whole thread. The research is admirable, but impractical.

    Here's the beginning of a simpler workflow:

    1. Use large, fast storage (e.g. eSATA or Thunderbolt), budget not withstanding.

    2. Adopt consistent, codified project naming conventions. (e.g. YYDD-CLIENTNAME-Project Name)

    3. Let Media Composer decide where to put the media, not the other way around. New folders are created after about 5000 files.

    4. Use MDV or MediaMover to bring a specific project's media on- or offline.

    I think this further punctuates the need for Avid to come up with a more effective media management / project archival approach. Ongoing discussion can be found here.

    (Sidebar: if anyone's still using sparse disk bundles in the FCP X community, it won't be for long. See Consolidate.)

    Mid-2010 Mac Pro • 2018 MacBook Pro [view my complete system specs]

    isaact.co | Twitter | Instagram | Vimeo

     

    Writer: The Frame.io WORKFLOW Guide - http://workflow.frame.io/

     

    Post Workflow Consultant - https://isaact.co/

  • Thu, Jan 15 2015 2:03 AM In reply to

    Re: Anyone Fooling Around with Sparse Disc Bundles?

    Thanks, I'm glad someone's eyes have scanned across it.

     

    Certainly with my wall of text I can see how it comes across as impractical. When compared to just letting Avid do whatever it wants to, or using another platform where you can already just choose where media goes, or just buying specialist systems like ISIS to handle networked storage in an Avid environment it definitely is impractical.

     

    That said, I was quite verbose in explaining things to make sure the explanation is thorough and because I realised after I devised the methodology that it's much harder to explain than to do.

     

    I've tried Media Mover in the past and it's not bad, but I've had issues with it failing to notice items that are there and that Avid seems capable of noticing and alsoit doesn't help with keeping more than one project 'online' at a time. You could argue that you really shouldn't need to keep more than one online at a time per machine, especially editing over network for example which is the main circumstance where I'd want to do this, because it isn't supported and to do edit over networked storage one should use supported systems, software and hardware. That said, circumstances get in the way of the ideal and I've often had to rely on being able to do that. I've had systems editing over high bandwidth networks with standard switching equipment and just plain windows software and no specialist hardware or software to manage shared storage and it worked well enough for emergencies.

     

    For that reason, and because the Avd media management system of single Avid MediaFiles folder per volume being fraught with negative consequences I found it necessary to devise ways to make it work the way I have often needed it to which I think is also how many people would ultimately need it to in a large array of circumstances. There are methods for this in windows environments that are simpler than what I describe here in concept but in terms of steps to follow and difficulty, are really not that far from the methodology I'm proposing here for macs.

     

    The symlink and sparsebundle method sounds convoluted and given what the aim of it is for to begin with it is, but it's actually really not that difficult to do once you've done it once. Try it out, it's pretty logical, and so long as you name your sparsebundles meaningfully it's quite an intuitive way to operate. You can effectively forget what's happening in the background and just think of each sparsebundle as if it were actually a separate physical drive. The fact that it's just a conduit to funnel media in to a folder doesn't affect how the operator does business day to day in Avid.

     

    Ultimately though yes, since the very first day I ever used avid and still today I've thought the approach whereby you can not designate a destination for your media is a terrible, terrible idea and I have to say I think I've largely been right about that. It simply isn't practical, the existence of the media mover software is a testament to that.

  • Thu, Jan 15 2015 3:02 AM In reply to

    Re: Anyone Fooling Around with Sparse Disc Bundles?

    AvowedMediaComposer:
    I've thought the approach whereby you can not designate a destination for your media is a terrible, terrible idea and I have to say I think I've largely been right about that. It simply isn't practical,
    If you think about it keeping in mind how you know Avid works it is actually very simple.


    In a local storage setup Avid ALWAYS sends your media to the "1" folder
    At the end of your session (or Project if you only work on one at a time) go to the Avid MediaFiles\MXF folder and rename your "1" folder to whatever Alpha numeric name you want. (100 Projectname, Date Projectname as examples)

    This renamed folder has been indexed by Avid so will be read by any Avid worldwide, if you move that folder to that Avid's Avid MediaFiles\MXF\ folder.  It is simply the easiest and most robust Project based NLE media management system. All you need to add to this is (free) MDV to tidy up any forgetfulness on your part during the process. Wink

  • Thu, Jan 15 2015 3:15 AM In reply to

    • Robert B
    • Not Ranked
    • Joined on Sun, Aug 7 2011
    • Los Angeles
    • Posts 83
    • Points 1,005

    Re: Anyone Fooling Around with Sparse Disc Bundles?

    Just to chime in on this thread. I haven't been using the symlink method described above, but have had success using the sparsedisc bundle method for a few years now (probably six months before my post above). Our projects rarely exceed 2 TB, so we normally use one sparedisk bundle to hold all the camera media (digital masters) and the other to hold the Avid project, Avid media and any associated project files. So far no problems (knock on plastic). I've been able to access old projects just fine and it makes it easy to copy and archive whole projects without using Mediamover. 

    To simplify creating the sparsediskbundles - google creatediskimage app. 

    -Robert

     

    OFFICE: 3.5 GHz Core i7 Late 2013 Imac running OSX 10.11 with 32GB RAM. NVIDIA GTX 780M 4096 MB. Avid Media Composer 8.5/ Pro Tools 12.4 MOBILE: 17"... [view my complete system specs]
    Filed under:
  • Thu, Jan 15 2015 2:49 PM In reply to

    Re: Anyone Fooling Around with Sparse Disc Bundles?

    It's simple yes, in so far as by limiting your options you don't run in to problems caused by having those options present. In platforms where you can designate media you can have issues with manual relinking sometimes, or with people developing their own project structures that differ from person to person and facilitiy to faclility and you have issues whereby people do not take due care and have media distributed confusingly accross all sorts of places that causes all manner of problems later on down the pipe.

     

    Then again, removing or omitting functionality which has some serious advantages in the name of simplicity is a double edged sword. Sometimes it's absolutely the right decision, but you've got to strike a balance. Enough people disagree with me on this that I guess Avid and many others believe they've come down on the right side of that balance but I definitely do not believe that is the case in this instance.

     

    I guess my workflow described above is useful mainly if you need to keep multiple instances of an 'Avid MediaFiles' folder available to Avid Media Composer on a single volume. This is helpful for example as a crude form of shared network media management so that a client machine editing data from a storage pool attached to another machine via network interface can not interfere with another instance of Avid Media Composer running on the machine to which that storage is attached. It's also helpful for transferring media from one project to another that may both be temporarily residing on a single volume. If you know that you're going to move the media off of that volume later, it will be helpful to you to be able consolidate from one project to the other so that it's all available and ready when it is eventually moved off that storage. It's also a particularly good way to split a large batch of transcoding amongst networked machines, if you create a folder as a destination for each machine's portion of the batch, you can then use the sparsebundle symlink method to designate each of those folders as the respective destinations for each machine running Avid to transcode, allowing them all to simultaneously write to the same storage pool without corrupting the database. Once the batch is done you can just move the mxf media from each folder to the main Avid MediaFiles folder of the project, this way you don't have to wait for each machine to first complete it's batch locally, and then wait for the completed transcodes to transfer to the desired storage volume as the transcoding and transferring is done simultaneously by each machine. This also means that you don't have to worry about having enough space for a particularly large batch of transcodes for each machine you may divide the batch amongst. For observational documentary for example, this can be a lifesaver.

     

    I think most of the situations I describe are probably something that's more elgantly solved by simply, scheduling enough time in post for ingesting to be realistically completed, or using a supported centralised media server or ISIS system to implement proper management of shared media amongst Avid clients. You could also just have enough space for a project you've embarked upon to begin with which again should be a no brainer. Nevertheless though, you can easily find yourself in situation where people didn't schedule enough time and you've been lumped with achieving the impossible, or you can't afford the equipment and/or software necessary for shared media editing facilities; yet you still need their functionality, or where the best estimates of space requirements were just wrong and you've got to make do. These things unfortunately do happen and with other software it's easier to be creative and flexible in attempting to solve such problems when they arise (even though they shouldn't) - Avid just definitely isn't flexible. That's why even if Avid refuse to introduce the ability to designate where your media directories are, it's necessary for me to find a way for it to do so despite them because often it just has to work and there's no time.

     

     

     

     

  • Thu, Jan 15 2015 4:38 PM In reply to

    • Isaac T.
    • Top 150 Contributor
    • Joined on Mon, Jan 9 2006
    • New York / Knoxville, TN
    • Posts 742
    • Points 8,765

    Re: Anyone Fooling Around with Sparse Disc Bundles?

    Sorry, I can't let this go.

    AvowedMediaComposer:

    I think most of the situations I describe are probably something that's more elgantly solved by simply, scheduling enough time in post for ingesting to be realistically completed, or using a supported centralised media server or ISIS system to implement proper management of shared media amongst Avid clients. You could also just have enough space for a project you've embarked upon to begin with which again should be a no brainer.

    All of this falls under failing to plan. And failing to plan is planning to fail.

    Plan for ingest time. Plan for the right storage type (local or network). Plan for the right storage amount. There's the unknown. Embrace that idea too. Cross the bridge when you come to it.

    AvowedMediaComposer:

    Nevertheless though, you can easily find yourself in situation where people didn't schedule enough time and you've been lumped with achieving the impossible, or you can't afford the equipment and/or software necessary for shared media editing facilities; yet you still need their functionality, or where the best estimates of space requirements were just wrong and you've got to make do.

    This is a human / business problem, more pointedly a customer relationship management problem.

    If you were hired as the expert, be the expert. If you're part of the team from the beginning, be in the room when those decisions are made. If it's impossible and you were brought in to save the day, be up front. Tell them it's impossible then craft a solution around this notion,

    "Fast, cheap, and good - you can only pick two."

    AvowedMediaComposer:

    These things unfortunately do happen and with other software it's easier to be creative and flexible in attempting to solve such problems when they arise (even though they shouldn't) - Avid just definitely isn't flexible. That's why even if Avid refuse to introduce the ability to designate where your media directories are, it's necessary for me to find a way for it to do so despite them because often it just has to work and there's no time.

    And this is why I can't let it go. This whole treatise, while admirable, betrays a fundamental lack of understanding in Avid's products.

    Avid media storage, yes, it's not like Premiere or FCP. But if you work against it, you're just asking for trouble.

    Sparse bundles were a hack for FCP X (now obsolete by libraries and consolidation), and they're a crazy-stupid hack for Avid users. If Apple changes some fundamental underpinning or, worse, yanks support for them, you're cooked.

    Whatever your role in this project, the goal is to keep people editing. If they have to wait for some Rube Goldberg sequence of events to start and end that process, your users will resent you. Or, they'll look for a new expert.

    Mid-2010 Mac Pro • 2018 MacBook Pro [view my complete system specs]

    isaact.co | Twitter | Instagram | Vimeo

     

    Writer: The Frame.io WORKFLOW Guide - http://workflow.frame.io/

     

    Post Workflow Consultant - https://isaact.co/

  • Fri, Jan 16 2015 5:04 AM In reply to

    Re: Anyone Fooling Around with Sparse Disc Bundles?

    Yeah. You're right.

     

    I tried to stop myself from responding because I contributed to the thread for informational purposes and didn't want to be antagonising anyone or carrying on a debate. It's hard not to say anything though, not least so I can hopefully correct the impression that I favour Heath Robinson style fixes and workarounds or devices over good practice. I don't.

     

    I went down this path because I was previously in a situation that necessitated using shared folders in windows, as network volumes to trick avid in to writing to these 'volumes' as a form of media segregation. I had to use it mostly for reasons described above. I was in that situation for a variety of factors and a perfect storm of circumstances that left me powerless to do anything else. Basically, I should have walked away, but as I did not, I had to deal with it and roll with the punches. I would be more careful in future to avoid working in an environment where its possible that I can have no recourse when all the above failures in planning occur. 

     

    I should imagine it won't be the last time that happens to anybody though. I know if I were desperately looking for a solution when there's no other way, this at least provides the means in Mac OS. In that scenario I'd appreciate that someone had done this, so I'm glad it's documented here for posterity if some people think they need it. It won't compare to doing things under ideal circumstances but it isn't actually that bad a way of working if you feel you've no choice. It's not difficult to set up at the start of a project, and once it is, it's almost invisible to an operator and doesn't introduce any additional waiting. The best thing about it is that the sparsebundle itself is expendable, you can delete it and your project is still fine, everything is still there, you just don't have the option of separating your media in to designated media directories anymore and must go back to square 1 where you had to use default Avid behaviour.

     

    I wouldn't recommend building an infrastructure entirely dependent or based on my workaround because of the risks of the sparsebundle being something Apple one day changes, or drops support for, but if that did happen to you, it'd just mean you no longer had this workaround, nothing is lost because nothing is stored in the sparsebundle but a shortcut.

     

    But anyway, yes, better to just work right.

  • Tue, Oct 6 2015 10:26 AM In reply to

    • jschreyer
    • Not Ranked
    • Joined on Tue, Jun 18 2013
    • Posts 18
    • Points 210

    Re: Anyone Fooling Around with Sparse Disc Bundles?

    Nice read. 

    I recently got a project with gazillions of OMFI files exceeding the 5000 files per folder limit multiple times. Using disk images was the only way to get below that limit without installling new physical discs.

    Media Mover is nice, so is Unity and Edit Share and whatnot, but sparse bundles are a quite good method if you're on a budget.

     

  • Tue, Oct 6 2015 11:04 AM In reply to

    Re: Anyone Fooling Around with Sparse Disc Bundles?

    Ah I'm glad someone's using these. I've actually found myself using them quite often now because I've gotten so used to how quickly they can be set up that they're useful for all kind of purposes. Recently I found it a good way to create a folder of media for use in a grade so that you can select the 'link to' option when creating an AAF of a timeline to send to grade. For various reasons that can be necessary and this method allows you to neatly segregate just the media you want to use for the locked timeline with handles, this is also a helpful way to deliver bits and pieces of projects if their small enough that you can send media over the internet since you can create a nice folder just for purpose. You could do this by embedding but sometimes there's reasons not to. 

     

    I'm interested in how they helped you in your scenario though. In my circumstance it allowed me to segregate avid media in to any folder I chose and therefore I could name those folders such that they would identify their contents and make them instantly movable through finder copying. If they were already in folders with over 5000 files contents, how did you use the sparsebundle methodology to help you there and what was the issue? Was it becoming difficult to distribute the media in those folders becaues you couldn't idenitify outside of Avid? 

     

  • Mon, Oct 12 2015 9:21 PM In reply to

    • jschreyer
    • Not Ranked
    • Joined on Tue, Jun 18 2013
    • Posts 18
    • Points 210

    Re: Anyone Fooling Around with Sparse Disc Bundles?

    AvowedMediaComposer:

     

     If they were already in folders with over 5000 files contents, how did you use the sparsebundle methodology to help you there and what was the issue?

    Was it becoming difficult to distribute the media in those folders becaues you couldn't idenitify outside of Avid? 

     

     

    I got 2 hard disks from a client, one had 4 partitions. Both disks filled with OMFI for audio files which unlike MXF files can only be stored in one OMFI Media folder on the root of a drive.

    One had like 10.000 + files. Copying all 5 OMFI folders didn't work of course and didn't i didn't dare to partition my media drive nor did I wanted to buy a new HD just for that project.

    So I created a bunch of sparse bundles and distributed my files onto my new virtual drives. I also started to create bundles for each of my projects and moved the belonging Avid Media folders.

    This method works extremely well til this day, in my opinion way better than renaming folders or using MediaMover.

     

     

Page 1 of 2 (24 items) 1 2 Next >

© Copyright 2011 Avid Technology, Inc.  Terms of Use |  Privacy Policy |  Site Map |  Find a Reseller