Removing Duplicate Files?



  • Before I knew better, I used one Runtime folder for Default content and manually installed content. I'm trying to dedupe, now.

    Questions:

    1. If I delete duplicates or move files, will Poser 11 be able to "automagically" locate the files in their new locations, so long as they remain in a subdirectory of Runtimes?

    2. Are there any guides or best practices for doing this? Huge library. I will back everything up before I start, but any guidance or recommended tools would be great! Any that use CRC values etc?



  • I actually wrote a little dedupe BASH script, that de-dupes exact dupes to hard links within a single runtime. Poser won't resolve soft links which is too bad.

    I also split runtimes along (more or less) groupings that I would tend to use together. For instance, I would not use toon figures except in toon scenes. So toon stuff in its own runtime. This lets me copy only the runtimes I think I'll need onto my laptop so I don't crush it with too much.
    I have a "gen4" and "common" that I'm pretty much always using. common has non-toon "places and things". gen4 has all the Daz 4g stuff, which is massive and still useful (through fitting room, etc.)

    And when I de-dupe, I do it within a runtime, since I may or may not have that particular runtime with me. First time I de-duped, I didn't do that. Derp!

    This also allows me to rsync each runtime independently which speeds up the process since I'm generally only adding content to 1 or 2 in any session. Plus, I keep a "master copy".

    I keep a "master" of my runtimes on one of my hosted servers, such that when I'm traveling, I can always get to any of them as needed. Plus, offsite copy :)

    The command I use to dedupe (mac => *nix command) is rdfind (for redundant data find)

    From my script (this is in a loop with a list of runtimes):
    rdfind -ignoreempty true -makehardlinks true -makeresultsfile false "${runtime}/Runtime/"

    This saves a lot of space. Consider where the same PNG is used for mat files, material poses, props, and cr2's.



  • @newoski, did i understand it right, you have put your manual installed content in the same runtime as poser?
    And now you want to separate the items?
    I would say, you should deinstall poser.
    The routine deletes only what comes with poser.
    So, after poser is gone, you should have a runtime with your content.
    I have made the same mistake years ago and it worked.
    For safety reasons i would save the whole runtime first.



  • @marco yes. There is a bug in the mac installer (for years) where if the name of the path of a shared runtime was the one poser wanted to use, it simply blew your existing content away. I tracked it down to the lines in the BASH script that issued a "mv" command of the new content over existing content. In Unix mv a directory over existing does not merge. It replaces. The entire tree, starting at the root you're moving over. I even highlighted that line, shows the installer log where that command was issued. From Poser support: our installer does not delete anything. Yes. It did. For 3 major versions. Now I keep 2 backups, rsync'd. And separate runtimes. lesson learned.



  • @thoennes autsch, thats german and means " it hurts".
    Im on windows and there it works fine.
    Maybe it is a long shoot, but do you have an windows system at hand?
    If so, just install poser, copy your runtim into the folder and deinstall poser.
    Afterwards you should have a clean runtime you can use on mac.
    The great amount of time in mind you will ned to do it manually i think it is worth a try.



  • @marco I've learned over the years (painfully) to keep my runtimes very separate, organized, and well backed up :D
    I wrote a nifty "mass installation script" I gave it just about any content (even without proper runtime folder) and it unzips, reorganizes, tags (in EA on mac and linux), keeps track of what files were installed where, and keeps a log, allowing for uninstall (never had to use). you can give it a mass of zips. It even tries to figure out content category. At one point, it was tracking products from major vendors that I had purchased, and it would list what I had bought, if I had the files downloaded, installed, etc.
    But since I keep my runtimes well organized, I haven't had to go through that sort of "mass install". And the vendors give jack (and squat) by way of API to help. Web scraping. And rendo makes even that a PITA, with their brutally slow dynamic web site. At least DAZ makes things dynamic but their site seems faster since it downloads all the product IDs into local browser data store right when you log in. For some stuff, you don't even have to query them. The data is already local. It's why a your purchased list comes up so fast. It's already in your browser.



  • @thoennes, and i thought i was clever...
    I have all my zip files in different folders, named after a main theme.
    Like stuff for V4 is called V4Cloth, or V4Character and so forth.
    If i need a new runtime, i just throw the zip folder into winrar and let them do its thing.
    The only thing i have not figured out yet is how to track the items for updates.
    But most of the time i use the figures and there textures but build settings and props in C4D if needed.
    I have created a seperat runtime with all of my props, textures, characters and so on.



  • @marco Sounds a lot like my organization :)

    I used to organize like that, then I started tagging in extended attributes. I had ended up with problems of things that belonged in SP4 and V4 or props I could used for any gen4. So I started tagging. Now I just do a spotlight search for content tagged "poser, v4, accessory, clothing" (order not necessary). Since linux supports EA, I can search on my backup content servers the same way.
    I wonder how many long time users have ended up with a similar "better practices" setup? Wouldn't it be nice for this lore to be captured somewhere?



  • @thoennes Yes, my Mac did do that, but it always asked me if I was certain I wanted to do that. There's a simple solution. Just go one level deeper in the folder. I've done that from Poser 6 on to Poser 10 when I stopped using my Mac.



  • @marco said in Removing Duplicate Files?:

    @newoski, did i understand it right, you have put your manual installed content in the same runtime as poser?
    And now you want to separate the items?
    I would say, you should deinstall poser.
    The routine deletes only what comes with poser.
    So, after poser is gone, you should have a runtime with your content.
    I have made the same mistake years ago and it worked.
    For safety reasons i would save the whole runtime first.

    Very interesting idea! Wil give this a shot