Monday, February 21, 2011

What is the best way to do loops in JavaScript

I have stumbled into several methods of looping in JavaScript, what I like the most is:

for(var i = 0; i < a.length; i++){
    var element = a[i];
}

But as tested here (http://www.robertnyman.com/2008/04/11/javascript-loop-performance/), it should probably be written so that the length is only calculated once.

In jQuery there is a .each that you can stick a function. I like this a little better, because I don't have to type the array twice, like in the above solution.

If JavaScript supported macros it would be a piece of cake to roll your own, but sadly it does not.

So what do you guys use?

From stackoverflow
  • You could just always use a while loop, and compute the array limit before hand.

    Var max = a.length-1;
    var i = 0;
    
    while(i <= max)
    {
    var element = a[i];
    i++;
    }
    
    Anders Rune Jensen : Sorry to be blunt, but I fail to see how this is an improvement. You still specify the array twice, and in your solution the environment around is polluted with two variables which scrope should not transcend the while loop construct.
    Jason Etheridge : The environment is going to be "polluted" anyway, given that in Javascript the scope of a variable is the enclosing function, not the block in which it is declared.
    Kibbee : Testing it against the link given in the question, it's at least as fast, if not faster, than any of the for loop implementations.
  • I've started using iterators where relevant. Performance is reasonable, however more importantly it allows you to encapsulate the looping logic:

    function createIterator(x) {
        var i = 0;
    
         return function(){
           return x[i++];
        };
    }
    

    Then to use:

    var iterator=createIterator(['a','b','c','d','e','f','g']);
    
    iterator();
    

    returns "a";

    iterator();
    

    returns "b";

    and so on.

    To iterate the whole list and display each item:

    var current;
    
    while(current=iterator())
    {
        console.log(current);
    }
    

    Be aware that the above is only acceptable for iterating a list that contains "non-falsy" values. If this array contained any of:

    • 0
    • false
    • ""
    • null
    • NaN

    the previous loop would stop at that item, not always what you want/expect.

    To avoid this use:

    var current;
    
    while((current=iterator())!==undefined)
    {
       console.log(current);
    }
    
    Anders Rune Jensen : Yeah, I have also been thinking about using iterators. They much better encapsulate the concept of traversing something than simple loops do. But how would you in your example print all elements in the iterator?
    Jason Bunting : Beautiful closure sweetness. Ahh, I loves 'em.
    Kibbee : How do you know when you've reached the end if your iterator doesn't have a HasNext function? If you just keep on calling "iterator" per your example, you will eventually get array index out of bounds.
    Ash : Kibbee, Anders, I've added a simple example of how to iterate the whole list to the answer.
    Anders Rune Jensen : Love it. Thanks!
    Kibbee : Sorry, I guess i'm a little to conditioned to using languages that throw exceptions when you try to access past the end of the array to try to code something that relies on the fact that JS doesn't throw an exception when you try to do this.
    Ash : Kibbee, No need to be sorry, I should have added that example earlier. One of the most important things to know in Javascript is the "falsy" values (values that are evaluated as false). These are: "", null, 0, NaN, false and finally undefined. The iterator relies on undefined being returned.
    insin : If you liked this answer, you may be interested in http://bob.pythonmac.org/archives/2005/07/06/iteration-in-javascript/ and Mochikit's http://mochikit.com/doc/html/MochiKit/Iter.html
    Manuel Ferreria : This is rather hackish. Me like it!
    Jani Hartikainen : Welcome to the world of Slow and Confusing. This idiom is *far* from common in JavaScript world, so people will have difficulties with your codebase if you decide to use this. If you have large loops, this approach is also *very* slow. Function calls are expensive in JS.
    Ash : @Jani, welcome to the world of New Ideas! This answer might help make the idiom more common. Also, performance is more than acceptable for normal day to day usage in my experience, and this depends heavily on the Javascript engine running it anyway. Function calls may be expensive, but un-maintainable Javascript costs a hell of a lot more!
    Jani Hartikainen : A "standard" loop is hardly unmaintainable. However I can see you understood my point ;)
  • Small improvement to the original, to only calculate the array size once:

    for(var i = 0, len = a.length; i < len; i++){ var element = a[i]; }
    

    Also, I see a lot of for..in loops. Though keep in mind that it's not technically kosher, and will cause problems with Prototype specifically:

    for (i in a) { var element = a[i]; }
    
    Vincent Robert : for..in loops are used to iterate over object properties, while they seem to work for Arrays, they will also iterate over the 'length' property or any other dynamically added property. That's why it does not work well with Prototype.
  • Just store the length in a variable first.

      var len = a.length;
      for (var i = 0; i < len; i++) {
        var element = a[i];
      }
    
  • And you can see a little test on the issue in: http://stackoverflow.com/questions/157260/whats-the-best-way-to-loop-through-a-set-of-elements-in-javascript#161664

  • If you have many elements in the array and speed is an issue then you want to use a while loop that iterates from highest to lowest.

      var i = a.length;
      while( --i >= 0 ) {
        var element = a[i];
        // do stuff with element
      }
    
    Paul Hargreaves : the >= 0 isn't needed, just change --i to i--
  • I don't use it myself, but one of my colleagues uses this style:

    var myArray = [1,2,3,4];
    for (var i = 0, item; item = myArray[i]; ++i) {
        alert(item);
    }
    

    like Ash's answer, this will hit issues if you've got "falsey" values in your array. To avoid that problem change it to (item = myArray[i]) != undefined

  • I know I'm late to the party, but I use reverse loops for loops that don't depend on the order.

    Very similar to @Mr. Muskrat's - but simplifying the test:

    var i = a.length, element = null;
    while (i--) {
      element = a[i];
    }
    
    annakata : well I'm *very* late to the party, but this is the correct answer and should be accepted as such. For the uninitiated, the i-- clause saves a comparison (because 0 = false in JS tests). Caveat 1: reverse order! Caveat 2: readability isn't great. Caveat 3: a cached for loop is very nearly as good
  • I tried Ash's solution and sadly declaring the two variables is quite tiresome in the end. I'm currently using the .each() from jquery and so far it's the best I have found. The only problem is that one can't return directly from the loop, and that break and continue is implementated as ugly idiosyncrasies (return true/false).

  • http://blogs.sun.com/greimer/entry/best_way_to_code_a

    That basically covers the whole subject

    Anders Rune Jensen : pretty wierd that the "while (i--)" solution is the fastest ;-)
  • So, first you identify the perfect javascript loop, I believe it should look like this:

    ary.each(function() {$arguments[0]).remove();})

    This may require the prototype.js library.

    Next, you get disgustet with the arguments[0] part and have the code be produced automatically from your server framework. This works only if the ladder is Seaside.

    Now, you have the above generated by:

    ary do: [:each | each element remove].

    This comes complete with syntax completion and translates exactly to the above javascript. And it will make people's head spin that haven't used seasides prototype integration before, as they read your code. It sure makes you feel cool, too. Not to mention the gain in geekiness you can get here. The girls love it!

  • I don't see what the problem with using a standard for(;;) loop is. A little test

    var x;
    var a = [];
    // filling array
    var t0 = new Date().getTime();
    for( var i = 0; i < 100000; i++ ) {
        a[i] = Math.floor( Math.random()*100000 );
    }
    
    // normal loop
    var t1 = new Date().getTime();
    for( var i = 0; i < 100000; i++ ) {
        x = a[i];
    }
    
    // using length
    var t2 = new Date().getTime();
    for( var i = 0; i < a.length; i++ ) {
        x = a[i];
    }
    
    // storing length (pollution - we now have a global l as well as an i )
    var t3 = new Date().getTime();
    for( var i = 0, l = a.length; i < l; i++ ) {
        x = a[i];
    }
    
    // for in
    var t4 = new Date().getTime();
    for( var i in a ) {
        x = a[i];
    }
    
    // checked for in
    var t5 = new Date().getTime();
    for( var i in a ) {
        if (a.hasOwnProperty(i)) {
            x = a[i];
        }
    }
    
    var t6 = new Date().getTime();
    var msg = 'filling array: '+(t1-t0)+'ms\n'+
              'normal loop: '+(t2-t1)+'ms\n'+
              'using length: '+(t3-t2)+'ms\n'+
              'storing length: '+(t4-t3)+'ms\n'+
              'for in: '+(t5-t4)+'ms\n'+
              'checked for in: '+(t6-t5)+'ms';
    console.log( msg );
    

    results in:

    filling array: 227ms
    normal loop: 21ms
    using length: 26ms
    storing length: 24ms 
    for in: 154ms
    checked for in: 176ms
    

    So:- for in's take the longest, using the length property (which is a property and doesn't need to be calculated) is nearly as fast as storing it first - which is only a whisker slower than using an integer.
    AND a for() is the usual way to loop over an array, which everyone expects and understands.

    All of them add a variable to the scope they run in - i - which is a common name for this use and so shouldn't be used for other things. Storing the length first adds another var - l - to the scope, which is unnecesary

How do you make php's json_decode compatible with firefox's javascript?

it appears if you have something like

var my_var = {"foo" : "bar"};

in javascript (with firefox at least) and post it to a php server you will receive a string like

{foo:"bar",}

on the server side. But json_decode in php doesn't like the trailing ',' or the lack or quotes around 'foo'. Is there a nice way to clean up the received json string?

The json object is sent with the drupal module json services.

EDIT: This question can be closed. The badly formed json is due to badly written js in the drupal module

From stackoverflow
  • What code are you using to POST the data? Make sure you're using something like json2.js.

    Steven Noble : I should check. I'm using drupal's json_service modulue.
    Steven Noble : Ah, this seems to be the crux of the problem. A badly written json encoder
  • This would be a good way to handle JSON where you can't predict whether there's extraneous commas:

    function json_decode_($json_string) {
        $json_string = preg_replace('/,(\s*)}/s', '$1}', $json_string);
        return json_decode($json_string);
    }
    

    Note, this is untested, but I think it should work.

How to scan a folder in Java?

How can I get a tree of all the files from a current folder in Java?

From stackoverflow
  • import java.io.File;
    public class Test {
        public static void main( String [] args ) {
            File actual = new File(".");
            for( File f : actual.listFiles()){
                System.out.println( f.getName() );
            }
        }
    }
    

    It displays indistinctly files and folders.

    See the methods in File class to order them or avoid directory print etc.

    http://java.sun.com/javase/6/docs/api/java/io/File.html

    Michael Myers : Your anchor link is broken (I guess the markup system assumes parentheses can't be in hyperlinks).
    OscarRyz : Thanks. What about now?
    Lipis : hehe.. I should probably RTFM more often.. :)
    Michael Myers : If you put the link in a footnote, the one you had originally should actually work. (I know because I just did it in a different question.)
    Michael Myers : Actually, just putting angle brackets around it ought to do the trick.
  • Check out Apache Commons FileUtils (listFiles, iterateFiles, etc.). Nice convenience methods for doing what you want and also applying filters.

    http://commons.apache.org/io/api-1.4/org/apache/commons/io/FileUtils.html

  • Not sure how you want to represent the tree? Anyway here's an example which scans the entire subtree using recursion. Files and directories are treated alike. Note that listFiles() returns null for non-directories.

    public static void main(String[] args) {
        final Collection<File> all = new ArrayList<File>();
        addFilesRecursively(new File("."), all);
        System.out.println(all);
    }
    
    private static void addFilesRecursively(File file, Collection<File> all) {
        final File[] children = file.listFiles();
        if (children != null) {
            for (File child : children) {
                all.add(child);
                addFilesRecursively(child, all);
            }
        }
    }
    
    marcospereira : I can't remember how much times I have wrote this code. :-P
    volley : Yeah it's like a recurring nightmare.. :P~
    Lipis : I have to accept this answer since I asked for the tree (I had accepted the Oscar Reyes' answer first).. even though adding one more line for the recursion wasn't that hard :)
  • In JDK7, "more NIO features" should have methods to apply the visitor pattern over a file tree or just the immediate contents of a directory - no need to find all the files in a potentially huge directory before iterating over them.

  • You can also use the FileFilter interface to filter out what you want. It is best used when you create an anonymous class that implements it:

    import java.io.File;
    import java.io.FileFilter;
    
    public class ListFiles {
        public File[] findDirectories(File root) { 
            return root.listFiles(new FileFilter() {
                public boolean accept(File f) {
                    return f.isDirectory();
                }});
        }
    
        public File[] findFiles(File root) {
            return root.listFiles(new FileFilter() {
                public boolean accept(File f) {
                    return f.isFile();
                }});
        }
    }
    

Index of Linq Error

If I have the following Linq code:

context.Table1s.InsertOnSubmit(t);
context.Table1s.InsertOnSubmit(t2);
context.Table1s.InsertOnSubmit(t3);

context.SubmitChanges();

And I get a database error due to the 2nd insert, Linq throws an exception that there was an error. But, is there a way to find out that it was the 2nd insert that had the problem and not the 1st or 3rd?

To clarify, there are business reasons that I would expect the 2nd to fail (I am using a stored procedure to do the insert and am also doing some validation and raising an error if it fails). I want to be able to tell the user which one failed and why. I know this validation would be better done in the C# code and not in the database, but that is currently not an option.

From stackoverflow
  • Comment out the first and third inserts to eliminate them as suspects.

    My first thought is that the second insert has the same ID as the first, but it's tough to diagnose your problem without more details about the error.

    NotDan : See my comments above. I am causing the insert to fail in the SP based on some conditional logic. I just want to know which one is failing.
  • You can specify explicitly a conflict mode like this one :

    context.SubmitChanges(ConflictMode.ContinueOnConflict);
    

    if you want to insert what is valid and not fail on the first conflict, then use the

    context.ChangeConflicts
    

    collection to find out which objects conflicted during the insertion.

What is an efficient way to check the precision and scale of a numeric value?

I'm writing a routine that validates data before inserting it into a database, and one of the steps is to see if numeric values fit the precision and scale of a Numeric(x,y) SQL-Server type.

I have the precision and scale from SQL-Server already, but what's the most efficient way in C# to get the precision and scale of a CLR value, or at least to test if it fits a given constraint?

At the moment, I'm converting the CLR value to a string, then looking for the location of the decimal point with .IndexOf(). Is there a faster way?

From stackoverflow
  • You can use decimal.Truncate(val) to get the integral part of the value and decimal.Remainder(val, 1) to get the part after the decimal point and then check that each part meets your constraints (I'm guessing this can be a simple > or < check)

  • System.Data.SqlTypes.SqlDecimal.ConvertToPrecScale( new SqlDecimal (1234.56789), 8, 2)
    

    gives 1234.67. it will truncate extra digits after the decimal place, and will throw an error rather than try to truncate digits before the decimal place (i.e. ConvertToPrecScale(12344234, 5,2)

    Pittsburgh DBA : 1234.57, but I voted you up anyway, because it's a great answer.

HOWTO: specify in app.config to call a function before Main() is called?

I really want to put in some sort of section handler into App.config that will execute some code before the application actually starts executing at Main. Is there any way to do such a thing?

From stackoverflow
  • Why dont you put that call as the first instruction in your main function?

    Otherwise you can define another entry point for your program and call your main from there but its basicaly the same

  • Not sure what you're trying to accomplish...but, I'm not aware of anyway you can have a console application run any other method before Main(). Why not do something like this:

    static void Main(string[] args)
    {
        //read your app.config variable
        callAlternate = GetConfigSettings(); 
        if(callAlternate)
            AltMain();
    
        ///...rest of Main()
    }
    

Why can't .NET parse a date string with a timezone?

.NET throws an exception trying to parse a datetime string in this format:

Fri, 10 Oct 2008 00:05:51 CST

Convert.ToDateTime("Fri, 10 Oct 2008 00:05:51 CST") results in an exception:

The string was not recognized as a valid DateTime. There is a unknown word starting at index 26

Character 26 obviously being the start of "CST"

In a quick test, PHP and javascript can both parse this string into a date with no problem. Is .NET just full of fail or is there a reasonable explanation?

From stackoverflow
  • http://msdn.microsoft.com/en-us/library/ey1cdcx8.aspx

    You need to use the overloaded DateTime.Parse to accurately parse timezones.

  • If a specific date and time format will be parsed across different locales, use one of the overloads of the ParseExact method and provide a format specifier.

.hide(), .show(), tables, and IE

I have some nested tables that I want to hide/show upon a click on one of the top-level rows.

The markup is, in a nutshell, this:

  <table>
    <tr>
      <td>stuff</td>
      .... more tds here
    </tr>
    <tr>
      <td colspan=some_number>
        <table>
        </table>
      </td>
    </tr>
  </table>

Now, I'm using some jQuery to target a link in the first table row. When that link is clicked, it pulls some data down, formats it as a bunch of table rows, and appends it to the table inside. Then it applies the .show() to the table. (this is all done via id/class targeting. I left them out of the sample for brevity).

This works beautifully in firefox. Click the link, data gets loaded, main table "expands" with the secondary table all nice and filled in.

Problem is -- Internet Explorer is giving me the finger. As best as I can tell, the data is getting appended to the inner table. The problem is that the .show() does not appear to be doing anything useful. To make matters more annoying, I've got a page that has this functionality that is working splendidly in both -- the only difference being two things:

In the one that is working, the inner table is wrapped in a div. I've even tried wrapping my table in this example in a div without success. In the one that is not working, I have an extra jQuery plugin loaded -- but I've removed this plugin and tried the page without it and it still fails to show the inner table.

I've tried attaching the .show to the parent tr, parent td, and the table itself with no success. I must be missing something incredibly simple, because as near as I can tell this should work.

Has anyone come across something like this before?

From stackoverflow
  • There are a bunch of bugs in IE related to modifying the contents of tables from Javascript. In a lot of cases, IE (including IE7) will even crash.

    I ran into this recently and I'm blanking on the work-around I came up with. Let me go back through my logs and see what I can find.


    OK, I found the case I ran into.

    I was doing something similar. I had a table and I was trying to add a <td> tag with a link in it via Javascript and that was causing a memory exception in IE7.

    In the end, the only way I could figure out to get around it was to rebuild the entire <table> in Javascript rather than trying to insert things into the existing one.

    To clarify, by rebuild I mean create a string containg the table HTML and add it to the innerHTML of a div. I'm not using DOM functions to create the table.

    Zack Mulgrew : I have read that IE will leak memory when tables are modified (presumably because it does not free removed and elements).
    Mark Biek : I could almost live with leaking memory if IE didn't crash outright :)
  • Keep in mind that the innerHTML of a <table> element is read-only in IE(6 for sure, not sure about 7). That being the case, are you explicitly adding a <tbody> element? If not, try adding one and adding the rows to the body element rather than the table element.

    <table>
      <tbody>
        <!-- Add stuff here... -->
      </tbody>
    </table>
    

    Microsoft info (sort-of) about this: PRB: Error Setting table.innerHTML in Internet Explorer Note: it says this is "by design".

    Leanan : Thank you so very much! I've been trying to puzzle this blasted thing out for over a day now! *bows*
    Zack Mulgrew : Good to know that this fixed it. I've had similar woes with tables, JavaScript, and IE before. Now, I always explicitly add a element just to be safe.
    Neall : innerHTML is flaky in general.

Where do you go for inspiration outside of technical arenas?

Maybe the title is not the best wording, and I haven't seen any similar questions (though I may have missed them).

Outside of 'keeping current' by reading tech news, Wired, etc, where do you go for non-technical inspiration on how to think about problems at hand in new/different ways?

For example, assembling a LEGO set, or doing an oil change might switch your mind around far enough that when you come back to a technical/implementation problem you're facing that you can think about it in a new/different way.

From stackoverflow
  • I play instruments, garden or do physical projects(ie, carving, modelling).

  • A reversed example would be like what Mel Bartholomew did for gardening when he developed Square Foot Gardening: he applied years of being a mechanical and process engineer to improving the planting efficiency of small gardens: why have two 4-foot-long rows of carrots when you could collapse them into a 1-by-2-foot square.

    Scottie T : My geometry's a little fuzzy. How can a square be 1x2?
    warren : ... each 'block' is a square foot, so if you have two next to each other, you have a square foot :)
    warren : times 2* (hit submit too fast)
  • Usualy when i walk the dog, I find solutions to my problems. Sometimes i can look different on problems then.

  • You can find inspiration in anything you do. The most important thing is getting your mind off of the usual drag of things you're used to, and take a look at something else entirely. Enjoying and relaxing. Inspiration often hits you unexpectedly but for most people i suppose it's in the shower. ;)

    Personally, i go for Astronomy and theoretical Physics to learn about the things our Universe holds in store for us. Another source of inspiration for me is, believe it or not, watching Mythbusters!

  • I usually get the most inspiration on a problem by stepping away, and trying to explain it to someone who doesn't know anything about it. That can engage different parts of my brain, that weren't actively used before.

    DarenW : i 2nd that. is also good to hear from people outside the specialty that they find it interesting. working too close to an exotic unique project, i forget how it looks to the general public or whoever is the end beneficiaries of the work.
  • alt text

    LEGO & inspiration

    Thomas Owens : I couldn't agree more.
    • Reading some ayn rand always steels me to keep trying on problems.
    • I love watching movies from a "how did they do this, or that" perspective.
    • I also read lots of mystery novels and sci-fi. Very creative stuff with very creative solutions.
    • Reading non-fiction (and not about solving the current problem!)
    • Listen to music (off all kinds)
    • Cook something (not a great cook, but it gets me up and standing, and moving about!)
  • Music, and moving around. One excellent way i found to clear my mind is Dances of Universal Peace, after which i'm much more clear thinking for a few days. The actions of learning strange words from an ancient language, remembering how to move, mindless repetition, and the joy of blending in harmony (yet not needing any actual talent) rebalances my brain. Also a great way to meet a variety of interesting people from around town, which also gets me out of technical ruts.

Hotkeys override

Difficult question. The answer is probably no, if all I found in the Intertubes is right, but it is worth a try. I need to override the CTRL+SHIFT+ESC and the CTRL+ESC combinations. It would be good to be able to override the WIN key combinations, but I have a low level hook that does such, I only wish I didn't need it. If I can manage to block the start menu and the task manager entirely by policy, the overrides will no longer be needed but I couldn't find the correct policy to do so.

From stackoverflow
  • So you need do override Hotkeys on your application? Then the answer is probably yes, how? No idea, but if this(http://www.inchwest.com/mapkeyboard.aspx) program can, so can you

    Leahn Novash : He does by using low level keyboard hooks. But those don't work with the CTRL+SHIFT+ESC and CTRL+ESC combinations.
  • You can override Ctrl+Esc (I just tried in AutoHotkey, it works), but not Ctrl+Shift+Esc, no more than Ctrl+Alt+Del.

    So using policies is probably the way to go... No experience in this field, I fear.

Java: How do I get the size of a java.sql.ResultSet?

Shouldn't this be a pretty straightforward operation? However, I there is no size() or length() method.

From stackoverflow
  • ResultSet rs = ps.executeQuery();
    int rowcount = 0;
    if (rs.last()) {
      rowcount = rs.getRow();
      rs.beforeFirst(); // not rs.first() because the rs.next() below will move on, missing the first element
    }
    while (rs.next()) {
      // do your standard per row stuff
    }
    

    KG : Inside the if(rs.last()) code block, wouldn't the correct method be rs.beforeFirst() instead of rs.first()? This way, you are not skipping the first record in your result set for processing in the while loop.
    JeeBee : KG - Indeed that looks right at a brief look at the code!
  • ResultSet.last() followed by ResultSet.getRow() will give you the row count, but it may not be a good idea as it can mean reading the entire table over the network and throwing away the data. Do a SELECT COUNT(*) FROM ... query instead.

    JeeBee : last() and getRow() aren't static methods in the ResultSet class.
    laz : For brevity's sake I always reference methods in this fashion when writing about them to others, regardless of whether they are static or not. Actually creating an instance of the object and calling the method is implied.
  • For a detailed description of how to connect to SQLServer database from Java, Statement, PreparedStatment, CallableStatement, ResultSet objects etc. check this link: HOW TO: SQL in JAVA.

Find which class in which jar has a given serialVersionUID

When I get a java.io.InvalidClassException, it gives me the serialVersionUID that it wants, and the serialVersionUID that it got. Is there an easy way to tell which of my dozens of jars using the wrong serialVersionUID?

Update: I should mention that our intention is to update everything at the same time, but I'm trying to debug a problem in our build and deploy process.

From stackoverflow
  • The best way to deal with this kind of trouble is to update jars on the server and client side at the same time. This will guarantee the same version of your classes on both sides and you'll not have trouble when serializing / deserializing. Tracking serial UIDs each time you have this problem is not going to solve anything, you're only going to waste considerable time and resources. Its much better to spend some time and implement a proper deployment / packaging strategy.

    If you really don't have any other choice, you can write a tool that loads a class from each jar (using a URLClassLoader) and then use java.io.ObjectStreamClass.getSerialVersionUID() to obtain the information you need.

  • clumsy but works

    i would use a reverse engineering tool
    1) unzip the jars
    2) run, say, jad on the class file tree
    3) run grep or any other find tool on this new tree of source to look for the serial version uid.

    Tom Hawtin - tackline : javap -private -verbose should work out the JDK box.
  • use the serialver tool from the sun jdk for each class in the jar.

  • Bear in mind that various JVM/container/classloader combinations have different opinions on which classes should be loaded from the bootclasspath versus the application/webapp classpath.

    This is complicated by the fact that serialver always loads from the bootclasspath first, so you may need to use -J-Xbootclasspath as below, to simulate different behaviour:

    f=/System/Library/Frameworks/JavaVM.framework/Versions/1.5/Classes/
    serialver -J-Xbootclasspath:.:$f/dt.jar:$f/classes.jar:$f/ui.jar javax.xml.namespace.QName
    

    Another approach is to use javap, for example:

    javap -verbose -bootclasspath . javax.xml.namespace.QName | sed -n -e '/static.*serialVersionUID/{N;p;}'
    

Why are pipes considered dangerous to use in Windows/unix/linux?

Why are pipes considered dangerous to use? What can be done to avoid these security issues?

I'm mostly interested in Windows, but if you have other OS information, please provide.

From stackoverflow
  • (assuming you're talking about Unix named pipes from the mention of 'c' and 'IPC'. Windows named pipes work somewhat differently)

    Anyone with permissions can write to a named pipe, so you have to be careful with permissions and locking (see flock()). If an application trusts the input it's getting from the named pipe (which will usually be the case unless you explicitly build input validation into it) then a malicious user can write any desired data into the named pipe if they have permission.

    Also, any user with permissions can read from the pipe and intercept data coming out of it if you have not exclusively locked it. The data is then missing from the input stream that the reader is expecting.

    James Curran : But doesn't this just boil down to "people with permission can do bad things", which is true with every OS feature?
    Adam Bellaire : That's true, but I think the most obvious failure case is having multiple processes running under the same user. They will all have the same permissions and access to the pipe, so you have to manage that access to prevent race conditions. e.g. TCP sockets, by contrast, are process-to-process.
    ConcernedOfTunbridgeWells : The main reason (and this apples to Windows named pipes to some extent as well) is that the pipes live in a public namespace, whereas sockets are private once a connection is established (assuming the TCP stream itself is not hijacked).

LIKE in Linq to SQL

I have a method that needs to accept an array of country names, and return a list of records that match one of those country names. I'm trying this

Public Shared Function GetConcessions(ByVal Countries As String()) As IEnumerable
    Dim CountryList As String = Utility.JoinArray(Countries) ' turns string array into comma-separated string
    Return (From t In New Db().Concessions _
                    Where CountryList Like t.Country _
                    Select t.ConcessionID, t.Title, t.Country)
End Function

but I get this error

  *Only arguments that can be evaluated on the client are supported for the LIKE method

In plain SQL, this would be simple:

 Select ConcessionID,Title from Concessions c where @CountryList like '%' + c.Country + '%'

How can I achieve this result in Linq to SQL?

Edit (clarification)

I get the same message with string.Contains. It would be fine with

t.Country.contains(CountryList)

but I need

CountryList.contains(t.Country)

and that throws the same error I listed above.

From stackoverflow
  • You can use SqlMethods.Like

    e.g.

    Where SqlMethods.Like(t.country, "%Sweden%")
    
    ctrlShiftBryan : very nice when u need to put the database field on the right side :)
  • I think what you want to do is construct a List from Countries and use

    List<string> ListOfCountries = new List(Countries)
    
    ...ListOfCountries.Contains(t.Country)
    

    This would translate into

    t.Country IN ('yyy','zzz',...)
    

    Please excuse my C#-ishness..

    leppie : +1 Wow! I was always wonder if LINQ used 'in'. Thanks :)
    Herb Caudill : Perfect - that works. Thanks.

Would you show things an Actor cannot do on a Use Case diagram?

On a Use Case diagram can you show things that an actor cannot do, for example because they won't have permissions to do it?

Or is it just implied due to the fact that they won't have a line joining them to the particular use case?

From stackoverflow
  • No. An Actor would be connected to everything that he can do. If the Actor can't do it, then it's not shown.

  • If the Use Case you are diagramming is the case where an actor attempts to do something that is not permitted and is then denied, then yes, I would show it.

    Otherwise, I would stick to only including things that are actually part of the use case.

  • You might model Role actors that can do the task. You could then have another use case that has the original actor attempting to acquire the given Role.

  • So from your answers are we saying that it depends on the 'system' that the use cases relate to?

    i.e. If we are modelling a web site we would only show what different role Actors do have access to but if we were modelling the permission system behind the web site we could show the requests for permissions being denied?

    Geoff : Yes, that is right.
    Geoff : As a note, make sure you aren't making your use cases too big. Keep them simple and don't try to include everything in one diagram.
  • This is what alternate paths are for. The basic path (a.k.a. happy path) will show what happens when the correct Actor initiates the Use Case. In the alternate paths you can show what happens if the wrong Actor attempts to initiate it.

PHP based form validation

I'm looking for an easy to use, reasonably complete form validation solution for php. I recall using one years ago that used a few tags on the HMTL side then captured the OB to replace them with some pretty serious code. This feels slightly like overkill me, yet I'm not REALLY wanting to go nuts with my own right now. I'm using SMARTY, so anything I use should not throw too many curve balls.

I'm going to be validating the basics, name, address, email, credit card number (format validation OK)...

Any ideas? Thanks folks!

From stackoverflow
  • Since you are already using Smarty, you may want to consider the PEAR package HTML_QuickForm (this page says it's being replaced by v2, but that's still in alpha. V1 is pretty solid).

    The package allows you to build and process your forms in an easy, straightforward API, and then has a renderer class that works with Smarty. Basically, you can spit out your entire form as an array of objects that Smarty can then render for you. You can control the look and feel with CSS. HTML_QuickForm can be configured to have client-side or server-side validation, or both. You just set up your rules for validation, and it just works. Hope that helps.

    Bruce the Hoon : A concern I have with this is that I need to very closely mirror a set of existing forms that are rather heavily styled. Do you think I would have any problems with this?

How to store images in your filesystem

Currently, I've got images (max. 6MB) stored as BLOB in a InnoDB table. As the size of the data is growing, the nightly backup is growing slower and slower hindering normal performance.

So, the binary data needs to go to the file system. (pointers to the files will be kept in the DB.)

The data has a tree like relation:

- main site
  - user_0
    - album_0
    - album_1
    - album_n
  - user_1
  - user_n
etc...

Now I want the data to be distributed evenly trough the directory structure. How should I accomplish this?

I guess I could try MD5('userId, albumId, imageId'); and slice up the resulting string to get my directory path:

  /var/imageStorage/f/347e/013b/c042/51cf/985f7ad0daa987d.jpeg

This would allow me to map the first character to a server and evenly distribute the directory structure over multiple servers.

This would however not keep images organised per user, likely spreading the images for 1 album over multiple servers.

My question is:
What is the best way to store the image data in the file system in a balanced way, while keeping user/album data together ?

Am I thinking in the right direction? or is this the wrong way of doing things altogether?

Update:
I will go for the md5(user_id0) string slicing for the split up on highest level. And then put all user data in that same bucket. This will ensure an even distribution of data while keeping user data stored close together.

  /var
   - imageStorage
     - f/347e/013b
       - f347e013bc04251cf985f7ad0daa987d
         - 0
           - album1_10
             - picture_1.jpeg
         - 1
           - album1_1
             - picture_2.jpeg
             - picture_3.jpeg
           - album1_11
             - picture_n.jpeg
         - n
           - album1_n

I think I will use albumId splitted up from behind (I like that idea!) as to keep the number of albums per directory smaller (although it won't be necessary for most users).

Thanks!

From stackoverflow
  • Just split your userid from behind. e.g.

    UserID = 6435624 
    Path = /images/24/56/6435624
    

    As for the backup you could use MySQL Replication and backup the slave database to avoid problems (e.g. locks) while backuping.

    Bill Karwin : Yep,that's what I was going to say. Reverse the digits in the numeric ID and it's more likely to distribute evenly, kind of round-robin.
    Alix Axel : @Bill: I don't get it. Why is reversing the userid more likely to distribute evenly? Is it because older users have had more time to upload more images?
    Bill Karwin : @Alix: Suppose 75 userid's are allocated in a monotonically increasing manner. The 1's digit cycles through 0 through 9, and repeats. On average, there are an equal number of occurrances of each digit. The 10's digit cycles too, but only 0 through 7; it never gets to 8 or 9. Also the 100's digit is just 0 -- no distribution at all. So it's better to use the lower digits of the userid as the index for the higher-level directories.
    Alix Axel : @Bill: Of course, makes perfect sense! Thanks for explaining it to me. =)
    wilmoore : @Bill, @Node: If the filename is hashed, should the directory structure still be derived from the un-hashed ID or is it better to apply the same strategy against the hashed value?
    Bill Karwin : @wilmoore: Depends on which hash algorithm you use, but probably you're using md5 or something, where any digit is as likely to be evenly distributed as another digit. So in that case there's no advantage to choosing the rightmost digits for your toplevel directories. You're just as likely to distribute files evenly by choosing the leftmost digits of the hash string.
  • Have you considered doing incremental backups of the DB?

    Jacco : I think that incremental backups would only temporarily solve the problem.
  • one thing about distributing the filenames into different directories, if you consider splitting your md5 filenames into different subdirectories (which is generally a good idea), I would suggest keeping the complete hash as filename and duplicate the first few chars as directory names. This way you will make it easier to identify files e.g. when you have to move directories.

    e.g.

    abcdefgh.jpg -> a/ab/abc/abcdefgh.jpg

    if your filenames are not evenly distributed (not a hash), try to choose a splitting method that gets an even distribution, e.g. the last characters if it is an incrementing user-id

    Jacco : Thanks, that sounds like a really useful tip.

How to map a network drive to be used by a service

Suppose some Windows service uses code that wants mapped network drives and no UNC paths. How can I make the drive mapping available to the service's session when the service is started? Logging in as the service user and creating a persistent mapping will not establish the mapping in the context of the actual service.

From stackoverflow
  • You wan't to either change the user that the Service runs under from "System" or find a sneaky way to run your mapping as System.

    The funny thing is that this is possible by using the "at" command, simply schedule your drive mapping one minute into the future and it will be run under the System account making the drive visible to your service.

    VoidPointer : the service does not run as "System". It is set up to run under a specific local account. Even I log in with that account, create a persistent network mapping, log out and restart the service, the mapping will not be available to the service.
  • You could us the 'net use' command:

    System.Diagnostics.Process.Start("net.exe", "use K: \\\\Server\\path");
    

    If that does not work in a service, try the Winapi and PInvoke WNetAddConnection2

    Edit: Obviously I misunderstood you - you can not change the sourcecode of the service, right? In that case I would follow the suggestion by mdb, but with a little twist: Create your own service (lets call it mapping service) that maps the drive and add this mapping service to the dependencies for the first (the actual working) service. That way the working service will not start before the mapping service has started (and mapped the drive).

    VoidPointer : with this mapping service setup, I think that the drive mapping established by the mapping service wouldn't be available in the context of the original service, would it?
    Treb : I think it should - there is only one environment for each winlogon session.
  • You'll either need to modify the service, or wrap it inside a helper process: apart from session/drive access issues, persistent drive mappings are only restored on an interactive logon, which services typically don't perform.

    The helper process approach can be pretty simple: just create a new service that maps the drive and starts the 'real' service. The only things that are not entirely trivial about this are:

    • The helper service will need to pass on all appropriate SCM commands (start/stop, etc.) to the real service. If the real service accepts custom SCM commands, remember to pass those on as well (I don't expect a service that considers UNC paths exotic to use such commands, though...)

    • Things may get a bit tricky credential-wise. If the real service runs under a normal user account, you can run the helper service under that account as well, and all should be OK as long as the account has appropriate access to the network share. If the real service will only work when run as LOCALSYSTEM or somesuch, things get more interesting, as it either won't be able to 'see' the network drive at all, or require some credential juggling to get things to work.

    VoidPointer : Very informative... Am I correct in assuming that logon scripts are also run only for interactive logon sessions and not for service sessions?
  • Hi,

    The reason why you are able to access the drive in when you normally run the executable from command prompt is that when u are executing it as normal exe you are running that application in the User account from which you have logged on . And that user has the privileges to access the network. But , when you install the executable as a service , by default if you see in the task manage it runs under 'SYSTEM' account . And you might be knowing that the 'SYSTEM' doesn't have rights to access network resources.

    There can be two solutions to this problem.

    1. To map the drive as persistent as already pointed above.

    2. There is one more approach that can be followed. If you open the service manager by typing in the 'services.msc'you can go to your service and in the properties of your service there is a logOn tab where you can specify the account as any other account than 'System' you can either start service from your own logged on user account or through 'Network Service'. When you do this .. the service can access any network component and drive even if they are not persistent also. To achieve this programmatically you can look into 'CreateService' function at http://msdn.microsoft.com/en-us/library/ms682450(v=vs.85).aspx and can set the parameter 'lpServiceStartName ' to 'NT AUTHORITY\NetworkService'. This will start your service under 'Network Service' account and then you are done.

    3. You can also try by making the service as interactive by specifying SERVICE_INTERACTIVE_PROCESS in the servicetype parameter flag of your CreateService() function but this will be limited only till XP as Vista and 7 donot support this feature.

    Hope the solutions help you.. Let me know if this worked for you .

    -Kushagra

Delphi 7 compile error - “Duplicate resource(s)” between .res and .dfm

I got a very similar error to the one below:

http://stackoverflow.com/questions/97800/how-can-i-fix-this-delphi-7-compile-error-duplicate-resources

However, the error I got is this:

  [Error] WARNING. Duplicate resource(s):
  [Error]   Type 10 (RCDATA), ID TFMMAINTQUOTE:
  [Error]     File P:\[PATH SNIPPED]\Manufacturing.RES resource kept; file FMaintQuote.DFM resource discarded.

Manufacturing.res is the default resource file (application is called Manufacturing.exe), and FMainQuote is one of the forms. .dfm files are plain text files, so I'm not sure what resources is being duplicated, how to find it and fix it?

If I tried to compile the project again, it works OK, but the exe's icon is different to the one I've set in Project Options using the "Load Icon" button. The icon on the app is some sort of bell image that I don't recognize.

From stackoverflow
  • Try renaming Manufacturing,res to Manufacturing.bak or something. Delphi should recreate the res file.

    You would of course need to recreate any references, strings etc in the res file in the new one, but worth trying anyway...

    Robo : Thought I tried that already, but I think I needed to restart Delphi or something to get it re-create correctly. The other problem I encountered which confused things a bit was Delphi 7 didn't like the 32 bit icon I used. Ran without errors when I used a simpler (but crappier looking) icon.
  • try looking for having an extra {$R *.res} or {$R *.dfm},you may have copied it from somewhere.

  • Delphi converts all of your DFM files into resources, and names them the name of the class. You can verify this by using a resource editor and opening any of your form based Delphi applications.

    search all of your units for instances of the TFMMAINTQUOTE form. Its most likely in two units, one of which is NOT linked to your project, but is being pulled in via the uses clause referencing the wrong unit (wrong as in it is saved with a different name but has the SAME form name, if it was part of your project then the compiler would complain when you added the unit in the first place).

    The simple solution to this problem is to find the TFMMAINTQUOTE form IN your project and rename the form to something else, but then the old TFMMAINTQUOTE will still be pulled in.

    I suggest using a good directory grep tool such as the one in GExperts to do your searching. It will save you alot of time and can be set to search your entire hard drive if so desired. The advantage of GExperts is that its free and integrates directly into the Delphi IDE.

    Erick Sasse : I can't live without GExperts Grep Search. ;)
    Robo : Thanks, I'll give GExperts a go, the ability to search/replace for the whole project directory is something I need.
  • I just had the same problem and found an extra {$R *.res} in there.

  • the extra {$R *.res} is in the *.dpr file like this:

    program Test;

    uses Forms, Unit1 in 'Unit1.pas' {Form1}, Sample in 'Sample.pas', Proc in 'Proc.pas';

    {$R *.res} //<----delete this if you put them in the Unt1.pas. ok.

    begin Application.Initialize; Application.CreateForm(TForm1, Form1); Application.Run; end.

  • If you rename a from and this form is referenced in the Uses part of other Units then you get the above error.

    Solution is a mixture of the above.

    (1) Change the resources file ending to .bak (so it will be re-created later). (2) Search through all the units and change the reference to old unit/form name to the new one. (3) re-compile and will now be ok.

  • I got the same error. I think that the contributing factors were: 1) no *.res-file 2) common units with another project 3) the project with the error had a form with the same name as a form in that another project

    Solution: rename the form (in the project with the error message)

  • Edit the RES file and delete from it the duplicate resource. This way you will be able to keep your original icon.