Thursday, February 3, 2011

Multiple foreign keys?

I've got a table that is supposed to track days and costs for shipping product from one vendor to another. We (brilliantly :p) stored both the shipping vendors (FedEx, UPS) with the product handling vendors (Think... Dunder Mifflin) in a "VENDOR" table. So, I have three columns in my SHIPPING_DETAILS table that all reference VENDOR.no. For some reason MySQL isn't letting me define all three as foreign keys. Any ideas?

CREATE TABLE SHIPPING_GRID(  
    id INT NOT NULL AUTO_INCREMENT PRIMARY KEY COMMENT 'Unique ID for each row',  
    shipping_vendor_no INT(6) NOT NULL COMMENT 'Foreign key to VENDOR.no for the shipping vendor (vendors_type must be 3)',  
    start_vendor_no INT(6) NOT NULL COMMENT 'Foreign key to VENDOR.no for the vendor being shipped from',  
    end_vendor_no INT(6) NOT NULL COMMENT 'Foreign key to the VENDOR.no for the vendor being shipped to',  
    shipment_duration INT(1) DEFAULT 1 COMMENT 'Duration in whole days shipment will take',  
    price FLOAT(5,5) NOT NULL COMMENT 'Price in US dollars per shipment lbs (down to 5 decimal places)',  
    is_flat_rate TINYINT(1) DEFAULT 0 COMMENT '1 if is flat rate regardless of weight, 0 if price is by lbs',  
    INDEX (shipping_vendor_no),  
    INDEX (start_vendor_no),  
    INDEX (end_vendor_no),  
    FOREIGN KEY (shipping_vendor_no) REFERENCES VENDOR (no),  
    FOREIGN KEY (start_vendor_no) REFERENCES VENDOR (no),  
    FOREIGN KEY (end_vendor_no) REFERENCES VENDOR (no)  
) TYPE = INNODB;

Edited to remove double primary key definition...

  • You defined the primary key twice. Try:

    CREATE TABLE SHIPPING_GRID(  
        id INT NOT NULL AUTO_INCREMENT PRIMARY KEY COMMENT 'Unique ID for each row',  
        shipping_vendor_no INT(6) NOT NULL COMMENT 'Foreign key to VENDOR.no for the shipping vendor (vendors_type must be 3)',  
        start_vendor_no INT(6) NOT NULL COMMENT 'Foreign key to VENDOR.no for the vendor being shipped from',  
        end_vendor_no INT(6) NOT NULL COMMENT 'Foreign key to the VENDOR.no for the vendor being shipped to',  
        shipment_duration INT(1) DEFAULT 1 COMMENT 'Duration in whole days shipment will take',  
        price FLOAT(5,5) NOT NULL COMMENT 'Price in US dollars per shipment lbs (down to 5 decimal places)',  
        is_flat_rate TINYINT(1) DEFAULT 0 COMMENT '1 if is flat rate regardless of weight, 0 if price is by lbs',  
        INDEX (shipping_vendor_no),  
        INDEX (start_vendor_no),  
        INDEX (end_vendor_no),  
        FOREIGN KEY (shipping_vendor_no) REFERENCES VENDOR (no),  
        FOREIGN KEY (start_vendor_no) REFERENCES VENDOR (no),  
        FOREIGN KEY (end_vendor_no) REFERENCES VENDOR (no)  
    ) TYPE = INNODB;
    

    The VENDOR primary key must be INT(6), and both tables must be of type InnoDB.

  • I ran the code here, and the error message showed (and it is right!) that you are setting id field twice as primary key.

  • Yeah, unfortunately that didn't fix it though. Now I'm getting:

    Can't create table './REMOVED MY DB NAME/SHIPPING_GRID.frm' (errno: 150)

    Doing a phpinfo() tells me this for mysql:

    Client API version 5.0.45

    Yes, the VENDOR.no is type int(6).

    From cmcculloh
  • Indeed, error 150 refers to a foreign key constraints error. Can you provide the definition of the VENDOR table?

    Is the PK of VENDOR an INT(6) or just an INT? From the above page:

    • The size and sign of integer types must be the same.
  • Can you provide the definition of the VENDOR table

    I figured it out. The VENDOR table was MyISAM... (edited your answer to tell me to make them both INNODB ;) )

    Thanks!

    (any reason not to just switch the VENDOR type over to INNODB?)

    From cmcculloh
  • MyISAM is more for debugging and testing... InnoDB is better for production. Check mysql.com for caveats, as there are some benefits/risks with using either.

    I'm actually having a similar problem - trying to declare 2 foreign keys in the same table (referencing two other tables)...

    The following gives a 1064 error, and no useable line number (which also confuses me!):

    CREATE TABLE users ( id INT NOT NULL AUTO_INCREMENT, username TEXT, PRIMARY KEY (id) ) ENGINE=INNODB DEFAULT CHARSET=utf8;

    CREATE TABLE profiles ( id INT NOT NULL AUTO_INCREMENT, userID INT NOT NULL, profileName TEXT, PRIMARY KEY (id), FOREIGN KEY (userID) REFERENCES users(id) ON UPDATE CASCADE ON DELETE CASCADE ) ENGINE=INNODB DEFAULT CHARSET=utf8;

    CREATE TABLE galleries ( id INT NOT NULL AUTO_INCREMENT, userID INT NOT NULL, galleryName TEXT, PRIMARY KEY (id), FOREIGN KEY (userID) REFERENCES users(id) ON UPDATE CASCADE ON DELETE CASCADE; ) ENGINE=INNODB DEFAULT CHARSET=utf8;

    CREATE TABLE photos (
    id INT NOT NULL AUTO_INCREMENT, userID INT NOT NULL, galleryID INT NOT NULL, photoName TEXT, PRIMARY KEY (id), FOREIGN KEY (userID) REFERENCES users(id) ON UPDATE CASCADE ON DELETE CASCADE, FOREIGN KEY (galleryID) REFERENCES galleries(id) ON UPDATE CASCADE ON DELETE CASCADE ) ENGINE=INNODB DEFAULT CHARSET=utf8;

    it's something to do with having more than 1 FK declaration... any ideas?? thanks!

Tabbed file browsing in Windows

I'm so tired of not having tabbed browsing when navigating through windows explorer.

I've tried a few solutions in the past, but what I really want is just an extension/addon/plugin/something that lets me browse my file system with tabs.

Does anyone have any good solutions that they have found to do this? Alternatively, I'm willing to just code it myself, but I don't even know where to begin... Is there any way to legally modify/customize the windows explorer? What language, how do I get started?

I can already visualize the replies "no, you're screwed, switch to linux" (which doesn't have tabbed browsing either by the way)...

  • X²Lite has tabs (there are near the bottom if you are looking at the screen shot on the front page)

    From Grant
  • Nice! Even appears to work with TortoiseSVN!

    Dang... Private or Academic use only... But only $30 to use at work... I just might buy it... Thanks!

    From cmcculloh
  • You could try QTTabBar. Didn't work too well for me personally, but there's others who swear by it.

    From Mun
  • I don't know of any add-ons. However, you can try Explorer XP, which has tabbed browsing.

    On another note, you can try ExplorerBreadCrumbs, which adds Vista like directory navigation for Windows XP.

  • TotalCommander

    Dual panel operation (like any xCommander), but with multiple tabs in each panel and A LOT of features. It's not free, costs $ 38 but (for me, at least) worths every penny.

    I've using it since 1998, and I couldn't live without it.

    I don't like sounding like an ad, but I'm a very satisfied user :))

    Nathan W : TotalCommander FTW. Best program in the world.
    From PabloG

Better Random Generating PHP

I know that just using rand() is predictable, if you know what your doing, and have access to the server.

I have a project that is HIGHLY dependent on choosing a random that is as unpredictable as possible. So I'm looking for suggestions, either other built in functions, or user functions that can generate a 'better' random number.

I used this to do a little test:

$i=0;
    while($i<10000)
     {
      $rand = rand(0,100);
     if(!isset($array[$rand]))
      {
      $array[$rand] = 1;
      }
     else 
      {
      $array[$rand]++;
      }
     sort($array);
     $i++;
     }

I found the results to be evenly distributed, and there is an odd pattern to the number of times each number is generated.

  • Variation on @KG, using the milliseconds since EPOCH as the seed for rand?

    KG : I like this suggestion waaaaaaaayyyyy better than mine :-)
    From jtyost2
  • Adding, multiplying, or truncating a poor random source will give you a poor random result. See Introduction to Randomness and Random Numbers for an explanation.

    You're right about PHP rand() function. See the second figure on Statistical Analysis for a striking illustration. (The first figure is striking, but it's been drawn by Scott Adams, not plotted with rand()).

    One solution is to use a true random generator such as random.org. Another, if you're on Linux/BSD/etc. is to use /dev/random. If the randomness is mission critical, you will have to use a hardware random generator.

  • http://random.org/clients/http/

    From Kevin
  • @Christian Lescuyer - I like random.org, but I decided to stay away from external calls, because if they go down so do I.

    I wote this quick, and it seems to produce more randomness.

    function myRand($min, $max)
     {
         $array = range($min, $max);
         shuffle($array);
         $i=0;
         while($i<10000)
          {
             $rand = rand($min, $max);
             $array[$rand] = rand($min, $max);
             shuffle($array);
             $i++;
          }
         return $array[rand($min, $max)];
     }
    
    From Unkwntech
  • By the way, I don't know why people say rand() is weak? It's been updated since PHP4 and seeds before every call.

    From Kevin
  • I would be wary of the impression of randomness: there have been many experiments where people would choose the less random distribution. It seems the mind is not very good at producing or estimating randomness.

    There are good articles on randomness at Fourmilab, including another true random generator. Maybe you could get random data from both sites so if one is down you still have the other.

    Fourmilab also provides a test program to check randomness. You could use it to check your various myRand() programs.

    As for your last program, if you generate 10000 values, why don't you choose the final value amongst the 10 thousand? You restrict yourself to a subset. Also, it won't work if your $min and $max are greater than 10000.

    Anyway, the randomness you need depends on your application. rand() will be OK for an online game, but not OK for cryptography (anything not thoroughly tested with statistical programs will not be suitable for cryptography anyway). You be the judge!

  • @Christian Lescuyer I agree about the impression of randomness, however if I run the loop with only 100 iterations I see almost an even distribution of 100 numbers between 1-100. With my myRand function I don't see that even distribution.

    From Unkwntech
  • @Unkwntech,

    Your myRand function will return a more random number, but is there a tradeoff in performance?

    Instead of 1 call to rand, you call it 20,001 times! not to mention the building of a "potentially" large array with 10,000 accesses.

    Also, as Kevin pointed out, PHP 4.2.0 and up auto-seed.

    From scunliffe

Haml: how do I set a dymanic class value?

I have the following html.erb code that I'm looking to move to Haml:

    <span class="<%= item.dashboardstatus.cssclass %>" ><%= item.dashboardstatus.status %></span>

What it does is associate the CSS class of the currently assigned status to the span.

How is this done in Haml? I'm sure I'm missing something really simple.

Edit: Gah. I can't believe I didn't figure that out on my own. Just goes to show that two sets of eyes are better than one. Thanks!

  • Not sure. Maybe:

    %span{:class => item.dashboardstatus.cssclass }= item.dashboardstatus.status

Split a string ignoring quoted sections

Given a string like this:

a,"string, with",various,"values, and some",quoted

What is a good algorithm to split this based on commas while ignoring the commas inside the quoted sections?

The output should be an array:

[ "a", "string, with", "various", "values, and some", "quoted" ]

  • What if an odd number of quotes appear in the original string?

    J c : That would imply an improperly quoted input string, so throwing an exception would be an option.
    From dulaneyb
  • Of course using a CSV parser is better but just for the fun of it you could:

    Loop on the string letter by letter.
        If current_letter == quote : 
            toggle inside_quote variable.
        Else if (current_letter ==comma and not inside_quote) : 
            push current_word into array and clear current_word.
        Else 
            append the current_letter to current_word
    When the loop is done push the current_word into array
    
    From Pat
  • The author here dropped in a blob of C# code that handles the scenario you're having a problem with:

    http://stackoverflow.com/questions/1898/csv-file-imports-in-net#6111

    Shouldn't be too difficult to translate.

    From Kev
  • If my language of choice didn't offer of way to do this without thinking then I would initially consider two options as the easy way out:

    1. Pre-parse and replace the commas within the string with another control character then split them, followed by a post-parse on the array to replace the control character used previously with the commas.

    2. Alternatively split them on the commas then post-parse the resulting array into another array checking for leading quotes on each array entry and concatanating the entries until I reached a terminating quote.

    These are hacks however, and if this is a pure 'mental' exercise then I suspect they are unhelpful. If this is a real world problem then it would help to know the language so that we could offer some specific advice.

    From Martin
  • I use this to parse strings, not sure if it helps here; but with some minor modifications perhaps?

    function getstringbetween($string, $start, $end){
    $string = " ".$string;
    $ini = strpos($string,$start);
    if ($ini == 0) return "";
    $ini += strlen($start);
    $len = strpos($string,$end,$ini) - $ini;
    return substr($string,$ini,$len);
    }

    $fullstring = "this is my [tag]dog[/tag]";
    $parsed = getstringbetween($fullstring, "[tag]", "[/tag]");

    echo $parsed; // (result = dog)

    /mp

  • Here's a simple algorithm:

    1. Determine if the string begins with a '"' character
    2. Split the string into an array delimited by the '"' character.
    3. Mark the quoted commas with a placeholder #COMMA#
      • If the input starts with a '"', mark those items in the array where the index % 2 == 0
      • Otherwise mark those items in the array where the index % 2 == 1
    4. Concatenate the items in the array to form a modified input string.
    5. Split the string into an array delimited by the ',' character.
    6. Replace all instances in the array of #COMMA# placeholders with the ',' character.
    7. The array is your output.

    Heres the python implementation:
    (fixed to handle '"a,b",c,"d,e,f,h","i,j,k"')

    def parse_input(input):

    quote_mod = int(not input.startswith('"'))

    input = input.split('"')
    for item in input:
    if item == '':
    input.remove(item)
    for i in range(len(input)):
    if i % 2 == quoted_mod:
    input[i] = input[i].replace(",", "#COMMA#")

    input = "".join(input).split(",")
    for item in input:
    if item == '':
    input.remove(item)
    for i in range(len(input)):
    input[i] = input[i].replace("#COMMA#", ",")
    return input

    # parse_input('a,"string, with",various,"values, and some",quoted')
    # -> ['a,string', ' with,various,values', ' and some,quoted']
    # parse_input('"a,b",c,"d,e,f,h","i,j,k"')
    # -> ['a,b', 'c', 'd,e,f,h', 'i,j,k']
  • This is a standard CSV-style parse. A lot of people try to do this with regular expressions. You can get to about 90% with regexes, but you really need a real CSV parser to do it properly. I found a fast, excellent C# CSV parser on CodeProject a few months ago that I highly recommend!

    MarkJ : There's also one in the .NET framework of course. Even though it's in Microsoft.VisualBasic you could still use it from C#. http://msdn.microsoft.com/en-us/library/microsoft.visualbasic.fileio.textfieldparser.aspx
    Simon Gillbee : Thanks! As a C# guy, I always forget that there are a bunch of useful VB libraries out there that I can use. Honestly, I think they are poorly named, since they are not really VB. They are just .NET.
  • Justin, that does not handle a string like this:

    input = '"a,b",c,"d,e,f,h","i,j,k"'

  • # Here's one in pseudocode (a.k.a. Python) in one pass :-P

    def parsecsv(instr):
    i = 0
    j = 0

    outstrs = []

    # i is fixed until a match occurs, then it advances
    # up to j. j inches forward each time through:

    while i < len(instr):

    if j < len(instr) and instr[j] == '"':
    # skip the opening quote...
    j += 1
    # then iterate until we find a closing quote.
    while instr[j] != '"':
    j += 1
    if j == len(instr):
    raise Exception("Unmatched double quote at end of input.")

    if j == len(instr) or instr[j] == ',':
    s = instr[i:j] # get the substring we've found
    s = s.strip() # remove extra whitespace

    # remove surrounding quotes if they're there
    if len(s) > 2 and s[0] == '"' and s[-1] == '"':
    s = s[1:-1]

    # add it to the result
    outstrs.append(s)

    # skip over the comma, move i up (to where
    # j will be at the end of the iteration)
    i = j+1

    j = j+1

    return outstrs

    def testcase(instr, expected):
    outstr = parsecsv(instr)
    print outstr
    assert expected == outstr

    # Doesn't handle things like '1, 2, "a, b, c" d, 2' or
    # escaped quotes, but those can be added pretty easily.

    testcase('a, b, "1, 2, 3", c', ['a', 'b', '1, 2, 3', 'c'])
    testcase('a,b,"1, 2, 3" , c', ['a', 'b', '1, 2, 3', 'c'])

    # odd number of quotes gives a "unmatched quote" exception
    #testcase('a,b,"1, 2, 3" , "c', ['a', 'b', '1, 2, 3', 'c'])
  • Looks like you've got some good answers here.

    For those of you looking to handle your own CSV file parsing, heed the advice from the experts and Don't roll your own CSV parser.

    Your first thought is, "I need to handle commas inside of quotes."

    Your next thought will be, "Oh, crap, I need to handle quotes inside of quotes. Escaped quotes. Double quotes. Single quotes..."

    It's a road to madness. Don't write your own. Find a library with an extensive unit test coverage that hits all the hard parts and has gone through hell for you. For .NET, use the free FileHelpers library.

    MarkJ : +10 if it would let me :)
    Magnus Smith : a great link on secretgeek - very amusing. but it only answers the question for those using .NET sadly.
    Judah Himango : True; though the advice applies to devs everywhere: don't role your own CSV parser. Ruby has one built-in, and there are libraries out there for Python, C++, most any widely used language.
    MarkJ : And although SecretGeek doesn't seem to know it, there's one built in to VB.NET too. http://msdn.microsoft.com/en-us/library/microsoft.visualbasic.fileio.textfieldparser.aspx
  • I just couldn't resist to see if I could make it work in a Python one-liner:

    arr = [i.replace("|", ",") for i in re.sub('"([^"]*)\,([^"]*)"',"\g<1>|\g<2>", str_to_test).split(",")]

    Returns ['a', 'string, with', 'various', 'values, and some', 'quoted']

    It works by first replacing the ',' inside quotes to another separator (|), splitting the string on ',' and replacing the | separator again.

    MarkJ : How do you know there aren't any | in the original string? What about escaping quotes inside quoted strings?
    From PabloG
  • Python:

    import csv
    reader = csv.reader(open("some.csv"))
    for row in reader:
        print row
    
    Alex. S. : I consider this as the best answer. Is exactly what I need!
  • What if an odd number of quotes appear in the original string?

    This looks uncannily like CSV parsing, which has some peculiarities to handling quoted fields. The field is only escaped if the field is delimited with double quotations, so:

    field1, "field2, field3", field4, "field5, field6" field7

    becomes

    field1

    field2, field3

    field4

    "field5

    field6" field7

    Notice if it doesn't both start and end with a quotation, then it's not a quoted field and the double quotes are simply treated as double quotes.

    Insedently my code that someone linked to doesn't actually handle this correctly, if I recall correctly.

    From ICR
  • Here's a simple python implementation based on Pat's pseudocode:

    def splitIgnoringSingleQuote(string, split_char, remove_quotes=False):
        string_split = []
        current_word = ""
        inside_quote = False
        for letter in string:
          if letter == "'":
            if not remove_quotes:
               current_word += letter
            if inside_quote:
              inside_quote = False
            else:
              inside_quote = True
          elif letter == split_char and not inside_quote:
            string_split.append(current_word)
            current_word = ""
          else:
            current_word += letter
        string_split.append(current_word)
        return string_split
    
    From Greg Bayer

Why do you participate in newsgroups/fora?

So why do you want to get involved with answering other people's questions?

I do it mostly to improve my own skills because I will see things that I might run into myself down the road. In other words helping others helps me indirectly by improving my skill set.

    1. Learning the answers to things I don't know.

    2. Enforcing/improving my knowledge of things I do know by teaching other people.

    From Mark Biek
  • Badges and reputation, of course. Helping others is great, learning is even better, but in the end, it's badges and reputation.

  • I love doing it, and always have. I get two mains benefits. The first is the buzz of knowing I am helping out other people, as well as helping them become just as passionate about development as I am. The second is the chance to increase my own knowledge, which is never a bad thing!

    From Martin
  • Here it's badges and reputation for some stuff but the desire to help others and gain an answer to my question for others.

    On things other than SO I find that the community can be important unless it's just to ask a single question before I vanish never to be seen there again.

    From Teifion
  • Would be nice if you could pick more than one as the answers since Mark and Martin's answers are very similar

    From SQLMenace
  • Like Martin and Mark, for me, answering questions has multiple benefits:

    1. It reinforces the information in my own head
    2. It allows me to experiment, to find the best way to verbalize the information
    3. It gives me sort of a "group-quid-pro-quo" feeling; I feel like if I help out enough on a website or in a forum, I'm more likely to receive help later if I need to ask; alternately, if I ask for help and receive it, I'm more likely to feel indebted to the site and hang around to see what help I can offer.

    Of course, having cool badges is nice too.

    From Adam V
  • David Taylor identifies our "secret" driving forces in his book The Naked Leader, this includes the following :

    • A sense of personal power and mastery over others
    • A sense of personal pride and, importance
    • Reassurance of self-worth and recognition of efforts
    • Peer approval and acceptance

    I think some of them apply for our motivations for participating in this newsgroup.

    From David

E-mail Notifications

In a .net system I'm building, there is a need for automated e-mail notifications. These should be editable by an admin. What's the easiest way to do this? SQL table and WYSIWIG for editing?

  • Are you just talking about the interface and storage, or the implementation of sending the emails as well?

    Yes, a SQL table with FROM, TO, Subject, Body should work for storage and, heck, a textbox or even maybe a RichText box should work for editing.

    Or is this a web interface?

    For actually sending it, check out the System.Web.Mail namespace, it's pretty self explanatory and easy to use :)

    From Adam Haile
  • From a high level, yes. :D The main thing is some place to store the templates. A database is a great option unless you're not already using one, then file systems work fine.

    WSIWIG editors (such as fckeditor) work well and give you some good options regarding the features that you allow.

    Some sort of token replacement system is also a good idea if you need it. For example, if someone puts %FIRSTNAME% in the email template, the code that generates the email can do some simple pattern matching to replace known tokens with other known values that may be dynamic based on user or other circumstances.

    From palehorse
  • I am thinking that if these are automated notifications, then this means they are probably going out as a result of some type of event in your software. If this is a web based app, and you are going to have a number of these being sent out, then consider implementing an email queue rather than sending out an email on every event.

    A component can query the queue periodically and send out any pending items.

    From Vaibhav
  • The queue is a great idea. I've been throwing around that type of process for awhile with my old company.

  • Adam Haile writes:

    check out the System.Web.Mail namespace

    By which you mean System.Net.Mail in .Net 2.0 and above :)

  • How about using the new Workflow components in .Net 3.0 (and 3.5)? That is what we use in combination with templates in my current project. The templates have the basic format and the the tokens that are replaced with user information.

Making a production build of a PHP project with Subversion

If you are working in PHP (or I guess any programming language...) and using subversion as your source control, is there a way to take your project (for example):

C:\Projects\test\.svn
C:\Projects\test\docs\
C:\Projects\test\faq.php
C:\Projects\test\guestbook.php
C:\Projects\test\index.php
C:\Projects\test\test.php

and build/copy/whatever it so it weeds out certain files and becomes:

C:\Projects\test\faq.php
C:\Projects\test\guestbook.php
C:\Projects\test\index.php

automagically? I'm getting tired of making a branch, and then going through the branch and deleting all of the ".svn" folders, the docs directory, and my prototyping files. I know I could probably use a .bat file to only copy the specific files I want, but I was hoping there was some way with subversion to sort of pseudo ignore a file, to where it will still version it, but where you could make a snapshot of the project that ignores the files you told it to psuedo ignore.

I know I read online somewhere about some functionality that at least lets you copy without the .svn folders, but I can't find it now...

  • Here's a wacky gum-and-shoestring idea:

    Copy all the files manually or using your existing method for the first time. Then, since I take it you're on a Windows platform, install SyncToy and configure it in the subscribe method, which would effectively one-way copy only the changes made since the last pseudo-commit to production for files already in production. If you want to add a file you can just copy it manually and resume the SyncToy operation.

  • If you use TortoiseSVN, you can use the export feature to automatically strip out all of the .svn files. I think other svn things have the same feature.

    Right click the root project folder, TortoiseSVN > Export and tell it where you want the .svn free directory.

    Frank V : The export feature *should* work with any client but namely the command line client; it is a standard feature.
    From Grant
  • Copy all the files manually or using your existing method...

    @saint_groceon: I like it. It is wacky gum-and-shoestring, but it's intriguing...

    TortoiseSVN > Export

    Ok, I think that's the method I must have been reading about online. Thanks!

    From cmcculloh
  • Ok, so my final solution is this:

    Use the export command to export to a folder called "export" in the same directory as a file called "deploy.bat", then I run the deploy script (v1 stands for version 1, which is what version I am currently on in this project) This script utilizes 7-Zip, which I have placed on my system path so I can use it as a command line utility:

    rem replace the v1 directory with the export directory
    rd /s /q v1
    move /y export\newIMS v1
    rd /s /q export

    rem remove the prepDocs directory from the project
    rd /s /q v1\prepDocs

    rem remove the scripts directory from the project
    rd /s /q v1\scripts

    rem remove individual files from project
    del v1\.project
    rem del v1\inc\testLoad.html
    rem del v1\inc\testInc.js

    SET /P version=Please enter version number:

    rem zip the file up with 7-Zip and name it after whatever version number the user typed in.
    7z a -r v%version%.zip v1

    rem copy everything to the shared space ready for deployment
    xcopy v%version%.zip /s /q /y /i "Z:\IT\IT Security\IT Projects\IMS\v%version%.zip"
    xcopy v1 /s /q /y /i "Z:\IT\IT Security\IT Projects\IMS\currentVersion"

    rem keep the window open until user presses any key
    PAUSE

    I didn't have time to check out the SyncToy solution, so don't take this as me rejecting that method. I just knew how to do this, and didn't have time to check that one out (under a time crunch right now).

    Sources:

    http://commandwindows.com/command2.htm
    http://www.ss64.com/nt/

    From cmcculloh
  • Ok, so my final solution is this:...

    Can you version the deploy.bat itself? Then you can have it self distruct when it's done. That's always fun.

    From Grant
  • Can you version the deploy.bat itself? Then you can have it self distruct when it's done. That's always fun.

    lol. Actually, the deploy.bat file sits inside of the "branches" directory in my project. I export the main copy that is under "trunk" into the "export" folder in "branches" (I know, technically I should be putting them in "tags" not "branches", that will be a slight tweak for the next project). So anyways, yes, the deploy.bat file is versioned...

    I guess what would make this all complete is somehow making a "export.bat" file that called the "export" command in tortoise and told it where to export to, and then called the "deploy.bat" file. Then I would have a one step build and deploy process for my project :)

    From cmcculloh

How do I edit work items in the Visual Studio 2008 xml editor?

I'm trying to customize some TFS work items via the VS2008 xml editor, but every time I open a work item xml file it jumps to the graphical designer. All that gives me is a "View XML" button that doesn't let you edit the xml directly.

  • Ah, looks like you have to go to File->Open and click the down arrow next to the Open button to "Open With" the xml editor. If someone wants to copy and paste this, free accepted answer :P

    From Luke
  • I don't have TFS but I know in regular VS there is an Open With... option in most items' contextual menu that even let you change the default editor. Very useful when you are tired of the Designer opening instead of the Code file on Windows forms.

    From Coincoin
  • As per Coincoin's answer, this feature is also great for setting the default editor for ASPX. If you want to go to the Code Editor most often, then this is a default you'd want to change.

    From Daniel O
  • Reading this - I think perhaps you don't realize - that there is no need to edit the XML - in fact it is very difficult to do so. The graphical designer will actually let you change the Work Item type, adding new fields, changing workflow, rules etc.

    The only reason to change the XML is if there's a bug in the Process Editor (the tool that gives the graphic designer). I have done extensive modifications of Work Item types and only had one instance where I had to change the XML.

    From fuzzbone

Memcached chunk limit

Why is there a hardcoded chunk limit (.5 meg after compression) in memcached? Has anyone recompiled theirs to up it? I know I should not be sending big chunks like that around, but these extra heavy chunks happen for me from time to time and wreak havoc.

Drag and drop ftp file upload web widgets

It seems like drag and drop upload widgets disappeared from the face of Web 2.0. The last one of these I remember using was an activex widget, and inability of using it in anything other than IE doomed it. Have you used or seen one of these recently?

  • FTP Drop for Yahoo Widgets allows you to drag files over the widget and the file will be sent to the defined ftp server.

    From GateKiller
  • The Dojo Toolkit JavaScript library supports some drag & drop functionality that I've seen work in IE6+ and FF2+. The nice thing about Dojo and other JS libraries is that they abstract away all of the browser detection stuff.

    I'm sure other JS libraries support this functionality.

    From Terrapin
  • Our current project makes heavy use of drag+drop, using GWT and gwt-dnd you can do some very cool stuff. Standards based, and works in IE6, Safari, Firefox, Opera, etc..

    The issue of how to transmit a file is a seperate one I beleive.

    As for FTP support, I see that as being mostly replaced with HTTP File Upload support.

    In the case you need more flexibility (progress bar, multiple file selection), then you can make use of flash to do this. You can use Javascript to interact with an invisible flash app which performs the file transfer. YUI's file upload control does this. You can see an example on Flickr's enhanced upload page.

    We've built a custom version designed for use with GWT apps. Same concept.

  • Hi,

    You can upload to FTP in browser with an applet such as JFileUpload. See: http://www.jfileupload.com/products/jfileupload/index.html

    It supports regular FTP, FTPS (explicit and implicit) and SFTP (FTP + SSH). It can resume broken transfer too.

    Applets can be moved outside browser since JRE 1.6.0_10.

    Cheers.

    From Uploader

Normalizing a Table with Low Integrity

I've been handed a table with about 18000 rows. Each record describes one customer location. The issue is, that when the person created the table, they did not add a field for "Company Name", only "Location Name," and one company can have many locations.

For example, here are some records that describe the same customer:

Location Table
+----+-------------------+
| ID | Location_Name     |
+----+-------------------+
| 1  | TownShop#1        |
| 2  | Town Shop - Loc 2 |
| 3  | The Town Shop     |
| 4  | TTS - Someplace   |
| 5  | Town Shop,the 3   |
| 6  | Toen Shop4        |
+----+-------------------+

My goal is to make it look like:

Location Table
+----+-------------+-------------------+
| ID | Company_ID  | Location_Name     |
+----+-------------+-------------------+
| 1  | 1           | Town Shop#1       |
| 2  | 1           | Town Shop - Loc 2 |
| 3  | 1           | The Town Shop     |
| 4  | 1           | TTS - Someplace   |
| 5  | 1           | Town Shop,the 3   |
| 6  | 1           | Toen Shop4        |
+----+-------------+-------------------+

Company Table
+------------+---------------+
| Company_ID | Company_Name  |
+------------+---------------+
| 1          | The Town Shop |
+------------+---------------+

There is no "Company" table, I will have to generate the Company Name list from the most descriptive or best Location Name that represents the multiple locations.

Currently I am thinking I need to generate a list of Location Names that are similar, and then and go through that list by hand.

Any suggestions on how I can approach this is appreciated.

@Neall, Thank you for your statement, but unfortunately, each location name is distinct, there are no duplicate location names, only similar. So in the results from your statement "repcount" is 1 in each row.

@yukondude, Your step 4 is the heart of my question.

  • I've had to do this before. The only real way to do it is to manually match up the various locations. Use your database's console interface and grouping select statements. First, add your "Company Name" field. Then:

    SELECT count(*) AS repcount, "Location Name" FROM mytable
    WHERE "Company Name" IS NULL
    GROUP BY "Location Name"
    ORDER BY repcount DESC
    LIMIT 5;

    Figure out what company the location at the top of the list belongs to and then update your company name field with an UPDATE ... WHERE "Location Name" = "The Location" statement.

    P.S. - You should really break your company names and location names out into separate tables and refer to them by their primary keys.

    Update: - Wow - no duplicates? How many records do you have?

    From Neall
  • Please update the question, do you have a list of CompanyNames available to you? I ask because you maybe able to use Levenshtein algo to find a relationship between your list of CompanyNames and LocationNames.


    Update

    There is not a list of Company Names, I will have to generate the company name from the most descriptive or best Location Name that represents the multiple locations.

    Okay... try this:

    1. Build a list of candidate CompanyNames by finding LocationNames made up of mostly or all alphabetic characters. You can use regular expressions for this. Store this list in a separate table.
    2. Sort that list alphabetically and (manually) determine which entries should be CompanyNames.
    3. Compare each CompanyName to each LocationName and come up with a match score (use Levenshtein or some other string matching algo). Store the result in a separate table.
    4. Set a threshold score such that any MatchScore < Threshold will not be considered a match for a given CompanyName.
    5. Manually vet through the LocationNames by CompanyName | LocationName | MatchScore, and figure out which ones actually match. Ordering by MatchScore should make the process less painful.

    The whole purpose of the above actions is to automate parts and limit the scope of your problem. It's far from perfect, but will hopefully save you the trouble of going through 18K records by hand.

    From jakemcgraw
  • I was going to recommend some complicated token matching algorithm but it's really tricky to get right and if you're data does not have a lot of correlation (typos, etc) then it's not going to give very good results.

    I would recommend you submit a job to the Amazon Mechanical Turk and let a human sort it out.

  • Ideally, you'd probably want a separate table named Company and then a company_id column in this "Location" table that is a foreign key to the Company table's primary key, likely called id. That would avoid a fair bit of text duplication in this table (over 18,000 rows, an integer foreign key would save quite a bit of space over a varchar column).

    But you're still faced with a method for loading that Company table and then properly associating it with the rows in Location. There's no general solution, but you could do something along these lines:

    1. Create the Company table, with an id column that auto-increments (depends on your RDBMS).
    2. Find all of the unique company names and insert them into Company.
    3. Add a column, company_id, to Location that accepts NULLs (for now) and that is a foreign key of the Company.id column.
    4. For each row in Location, determine the corresponding company, and UPDATE that row's company_id column with that company's id. This is likely the most challenging step. If your data is like what you show in the example, you'll likely have to take many runs at this with various string matching approaches.
    5. Once all rows in Location have a company_id value, then you can ALTER the Company table to add a NOT NULL constraint to the company_id column (assuming that every location must have a company, which seems reasonable).

    If you can make a copy of your Location table, you can gradually build up a series of SQL statements to populate the company_id foreign key. If you make a mistake, you can just start over and rerun the script up to the point of failure.

    From yukondude
  • @gomercobs, I appreciate your insight. I will attempt your suggestions and report back.

    I was able to find: http://www.codeguru.com/vb/gen/vb_misc/tips/print.php/c13137/ and it has been quite effective for me.

    Thank you.

    From Joe Mako
  • Yes, that step 4 from my previous post is a doozy.

    No matter what, you're probably going to have to do some of this by hand, but you may be able to automate the bulk of it. For the example locations you gave, a query like the following would set the appropriate company_id value:

    UPDATE  Location
    SET Company_ID = 1
    WHERE (LOWER(Location_Name) LIKE '%to_n shop%'
    OR LOWER(Location_Name) LIKE '%tts%')
    AND Company_ID IS NULL;

    I believe that would match your examples (I added the IS NULL part to not overwrite previously set Company_ID values), but of course in 18,000 rows you're going to have to be pretty inventive to handle the various combinations.

    Something else that might help would be to use the names in Company to generate queries like the one above. You could do something like the following (in MySQL):

    SELECT  CONCAT('UPDATE Location SET Company_ID = ',
    Company_ID, ' WHERE LOWER(Location_Name) LIKE ',
    LOWER(REPLACE(Company_Name), ' ', '%'), ' AND Company_ID IS NULL;')
    FROM Company;

    Then just run the statements that it produces. That could do a lot of the grunge work for you.

    From yukondude

What is the best way to store connection string in .NET DLLs?

The application my team is currently developing has a DLL that is used to perform all database access. The application can not use a trusted connection because the database is behind a firewall and the domain server is not. So it appears that the connection string needs to have a DB username and password. The DLL currently has the database connection string hard coded, but I don't want to do this when we launch as the assembly can be disassembled and the username and password would be right there in the open.

One of the requirements is that the password needs to be changed once every few months, so we would need to roll that out to our internal user base.

Is there a way to store the password encrypted in such a way we can easily distribute to the entire user base without storing it in the assembly?

Thanks in advance!

UPDATE: Thanks to everyone who's answered. I'll try to answer some of the questions back to me... The data DLL is used by both ASP.NET WebForms and VB.NET WinForms. I understand that Applications can have their own config files, but I haven't seen anything on config files for DLLs. Unfortunately, I can't get to the Jon Galloway post at work so I can't judge if that will work. From a development standpoint, we don't want to use web services inhouse, but may be providing them to third parties sometime next year. I don't think impersonation will work because we can't authenticate the user through the firewall. As a user (or former user) can be an attacker, we're keeping it from everyone!

  • I'm not certain, but I believe you can put it in a config file and encrypt the config file.

    Update: See Jon Galloway's post here.

    From Adam V
  • @proudgeekdad:

    Is this an ASP.NET app or a windows forms app running on multiple machines?

    If it's an ASP.NET app then just encrypt the connection strings section of your web.config.

    If it's a client app running on multiple machines, instead of storing the connection string locally, consider using a web service or some other kind of secure mechanism to store it centrally. This would facilitate easier updates in the future and you're not storing the connection string locally.

    Just some thoughts.

    Updated: @lassevk

    "Storing the connection string on a server, and obtaining it through a web connection sounds good, until you realize that you need security on that web connection as well, otherwise an attacker could just as well impersonate your program and talk to the web connection."

    Security on the web service was implicit. Depending on the type of deployment there are a numerous options...for example client side certificates.

    From Kev
  • I hate to say this but as soon as you put something on a client machine, security for that data goes out the window.

    If your program is going to decrypt that string, you need to assume that an attacker can do the same. Attaching a debugger to your program would be one way.

    Storing the connection string on a server, and obtaining it through a web connection sounds good, until you realize that you need security on that web connection as well, otherwise an attacker could just as well impersonate your program and talk to the web connection.

    Let me ask a question. Who are you hiding the connection string from? The user or an attacker? And if the user, why?

  • There are some other idea's also. You can always use impersonation. Also, you can use the Enterprise Library's (Common Library).

    <section name="enterpriseLibrary.ConfigurationSource" type="Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ConfigurationSourceSection, Microsoft.Practices.EnterpriseLibrary.Common, Version=3.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
    <enterpriseLibrary.ConfigurationSource selectedSource="Common">
    <sources>
    <add name="Common" type="Microsoft.Practices.EnterpriseLibrary.Common.Configuration.FileConfigurationSource, Microsoft.Practices.EnterpriseLibrary.Common, Version=3.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"
    filePath="Config\Exception.config" />
    </sources>

  • .NET supports encryption on config values like this. You could leave it in a config file, but encrypted.

  • You want to be able to distribute the DLL with all of the setup information being in a configurable place, but the fact is you can't have one of the handy-dandy .NET config files for a DLL unless you do something custom.

    Maybe you need to rethink what responsibility your DLL should have. Would it be possible, or make sense to require that the connection string be passed in by the user of your library? Does it really make sense that your DLL reads a config file?

  • Just assume that the bad guys WILL get the credentials out of your config file. This means that they'd be able to login to your database and do whatever that user is capable of. So just make sure that user can't do anything bad like access the tables directly. Make that user only capable of executing certain stored procedures and you'll be in better shape. This is one place that sprocs shine.

    Matt Blaine : I always try to limit the SQL login for web apps I've worked on in as many ways as possible.
    From Webjedi
  • Several options: 1. Store in web.config and encrypt. 2. Store in dll and obfuscate (dotfuscator). 3. Store one in web.config (encrypted of course) and rest in the database (if u have to use multiple and encryption/decryption becomes a pain).

    From