Why is my web application seeing latency?

We have an ASP.Net app, that behaves strangely under IIS6. The app itself is straightforward ASP.Net 2.0 Webforms deal, nothing too weird is going on in there (there are couple HTTP Modules in the pipeline, but I wouldn't consider those weird :) ). The thing I don't understand is the page execution times, or, more specifically, the difference between the time reported by ASP.Net tracing (trace.axd) and observed by the client (Fiddler). When the app is run on developer's box (WinXP, IIS5.1), the times reported by ASP.Net and Fiddler are very close:

Page exec time             :  0.0919834
Fiddler Total Sequence time:  0.1560980 

I can understand 60ms being spent getting 5KB worth of data from IIS to Fiddler (both of which run on the same machine, BTW). Now, when we move the code to the server (Win2k3, IIS6), the picture changes dramatically:

Page exec time             :  0.1702014
Fiddler Total Sequence time:  0.5156283 

This is same page, and Fiddler is again running on the same machine with the code. Why does it suddenly take 350ms to deliver the same 5KB?

PS. On both machines the results are obtained by accessing the URL via the actual machine's hostname, e.g. http://machinename/app/page.aspx (as opposed to http://localhost/app/page.aspx).

PPS. Configuration-wise, the setups of a dev box and the server are as close as we could make them -- both use the exact same web.config. Both hit the DB (sql server) with integrated auth, and, consequently, the app runs under a domain account. The app uses forms authentication, and does NOT impersonate (i.e. it always runs under the same account). Now, the way this works on IIS5 is different from IIS6 -- on IIS5 the account is specified in tag in the machine.config, and on IIS6 it's the AppPool setting. The setup seems pretty typical for both environments, and I can't imagine it causing 350ms delays...

From stackoverflow

• Do a trace route on the URL you are calling and compare them. I am betting with the developer machine you are staying internal to the machine, but on the production machine you are going external and then coming back in through the IP Address.

  If this is the case try adding this to your hosts file (c:\windows\system32\drivers\etc\hosts

  www.mysite.com    127.0.0.1
  

  This will make sure your request doesn't venture outside of the machine to make the request. You should see the response times to start to come in line with each other.

  Update

  Given the new updates. If the server is under load, while testing on production this could account for the difference, because it is actively trying to deliver more requests than the development machine which is only trying to deliver 1.

  Or it could be because you are testing two different version of IIS, 5.1 on XP and 6.0 on 2003. Really can't account for the differences unless the two environments are running the same software.

  : Nope. We are using IE7 to drive Fiddler in both cases, and since IE7 bypasses proxies on requests to localhost, we have to use the actual hostname in both cases.

  : BTW, what do you mean by "venture outside"? The only "venturing" I can think of is a DNS request, and it's all cached long before the time of the test...

  Nick Berardi : so you are actually calling http://localhost on your webserver and dev machine and getting these response differences? Because that changes you whole question and probably should have been mentioned.

  : We are calling http://machinename in both cases. However, even if it was http://localhost, I don't see how does it make a shred of a difference either way.

  Nick Berardi : Well it makes a difference if you are testing localhost vs your URL. That was my question to you. Since your are doing both by localhost, are you doing them while the server is processing real requests?

  : No, the server is idle. It's a dedicated "perf" box, that does nothing but run performance tests (we actually like to know how does the app perform before we push it into production).

  Nick Berardi : makes sense. just wasn't in the question so I had to ask. Have you tried setting up Windows 2003 on the developer machine to see how that performs. So that you are using the same version of IIS.

  : What's puzzling is not the raw performance per se, but the large difference between the times reported by ASP.Net and observed by Fiddler on W2k3.

• Is the app running in identical release configurations on both boxes?

  EDIT: The Request pipeline changed enormously between IIS5 and IIS6, trace.axd is only going to see the ASP.NET portion of it, not the new app pool and HTTP.Sys components.

  I'd imagine that configuration can be adjusted on IIS6 a bit, but you are probably looking at the difference of a lightweight non production webserver (IIS5) and a robust webserver with individual application pools to manage and more layers of abstraction.

  : Added PPS to the question. Basically, the setups are "as close as we could make them"

  : If this were the case, then EVERY .aspx page would take at least 350ms to come back. Which is obviously not the case.

• After expending one of the precious few support incidents we've got with our MSDN subscription, I finally know the correct answer to "where is all this time spent" question. In short, the time is spent in HTTP modules we have in our pipeline. The time measurements reported by ASP.Net trace.axd record only the time spent in the .aspx page itself, modules are NOT included.

  One easy way to ascertain this (and see how long does every module take to do its thing) is to use the ETW (Event Tracing for Windows). Here is the explanation (I strongly suspect that this post was written after they looked at our case :)) One thing I can add to the excellent description above is that I used the SvcTraceViewer instead of LogParser to analyze the trace output.

  Update: the above approach also works on Windows Server 2008, just make sure that you have Tracing installed.

MSSQL Server Management Studio (SSMS) 2005 New Query Template

How do I change a default "New Query" template in SSMS 2005?

From stackoverflow

• Posted this question as a reference, I did some googling, and found a simple way of doing it.

  All you need to do is to edit SQLFile.sql located here: <%= your SQL install dir %>\90\Tools\Binn\VSShell\Common7\IDE\SqlWorkbenchProjectItems\SQL\SQLFile.sql

  Also, Ctrl+Alt+T will show you all the other templates in SSMS; and check out Free SQL Server tools while you're at it.

  stack : You da man, thanks for putting this here!

• SUPER!!! Vielen Dank!

  Viele Grüße Lars

  roman m : you should make this a comment rather than an answer

Java AWT - BufferedImage problems when using J2ME and J9

Hey,

I'm trying to use the BufferedImage class in AWT. I'm using J2ME on IBM's J9 virtual machine.

When I try and call the BufferedImge.getRastor() method I get the following exception:

Exception in thread "main" java.lang.NoSuchMethodError: java/awt/image/BufferedImage.getRastor()Ljava/awt/image/WritableRaster;

Now, from what I know about the JVM that error is basically telling me that the BufferedImage class does not have a method called getRastor() which returns a WritableRaster object, however this method is documented in the API and it's from version 1.4.2 so should be compatable with J2ME.

I have no idea what is going on here, can you help?

Cheers,

Pete

From stackoverflow

• I don't think this is your answer... but since you quoted your exception and I assume you cut and pasted it, I'll try to help.

  Isn't the method:

  getRaster
  

  not

  getRastor
  

  ?

  (sorry if this is not what it is ailing you...)

• You won't be able to use anything from AWT in J2ME since its not supported.

  That happens because J2ME doesn't have AWT. AWT is intended to be used in desktop applications (Java SE), with a different user model and functionalities.

  You can take a look at J2ME docs here

  J2ME uses a different approach regarding GUIs, you may use the high level abstraction API (FORMS) and the low level API (CANVAS).

  Decio Lira : Maybe because your IDE has JavaSE on the classpath. I'm not sure.:/ What are you using? the sun's Wireless toolkit? Or some proprietary toolkit? Can you sucefully run this code on the actual device?

• The accepted answer is not correct (as of 2009-08-05). Exactly which API's you have available depends on the configuration and profile combination you deploy. However, you do need to refer to the J2ME documentation for exactly which 1.4 classes and members are supported.

  Using CDC 1.0 with Personal Profile 1.1 you get a cut-down version of Java SE 1.4. Personal Profile 1.0 is a cut-down Java SE 1.3. Both have a fairly complete implementation of AWT (no Swing, though).

  We actually use J9's CDC/PP on handheld/PDA devices quite successfully.

javascript for new tab (CTRL+T), new window (CTRL+N)?

When flash has keyboard focus, CTRL+T (new tab) and CTRL+N (new window) are intercepted by flash.

Is there a way to pass these events through to the browser so that they work (opening new tab, opening new browser) OR is there a javascript command for these actions?

From stackoverflow

• Closest you could get is to have ActionScript trigger Javascript to open a blank window to a blank URL

  // We abstract it in a function here in case we want to
  // change it later
  function openBlankWindow()
  {
     window.open( '' );
  }
  

  For most people, this will launch a new window or a new tab (depending on their browser preferences) but since it is being initiated by the web page, may be subject to pop-up blockers.

  There is no way to actually ask the browser to specifically do one of the two tasks you are asking about. I would be a security/annoyance nightmare if web pages had the permissions/privileges to do that.

  Daniel Lew : I don't think he wants to give Flash access to the browser so much as he wants normal browser functionality to work even inside of Flash.

  Peter Bailey : I know that, which is why the very first thing I said was "closest you could get". Allowing key-commands to bubble-up out of a flash movie to the browser is something only Adobe could fix.

  Daniel Lew : I was responding to your last paragraph, not the rest - the rest makes sense to me.

• This is a long standing issue with Flash and browsers. (And I mean long - check out this eight-year-old bug on Mozilla browsers.) The problem is that Flash intercepts all input events, rather than the browser. It's sandboxed in its own environment, and doesn't pass events back to the browser.

  Conceptually, this isn't necessarily a bad thing. What happens when Flash wants to listen to a "ctrl + n" event? Should the browser take focus away from Flash because it uses that hotkey already? It'd be a real pain for Flash developers, that is for sure.

  There have been proposals on how to fix this issue that I've seen for particular browsers, but there's no catch-all solution. For example, this solution is referenced in the bug, but it obviously won't work the way you want (since the user will have to jump through quite a few hoops to get it working).

  So... no, for now. Would be really neat if this problem could be fixed.

  Ben Blank : "Should the browser take focus away from Flash because it uses that hotkey already?" — Yes! Flash is a "guest" in the browser's "house". And a particularly ill-behaved one, too. :-)

  Daniel Lew : Hahaha, I've never heard the guest analogy before for plugins, that's pretty awesome.

• Peter Bailey is pretty much right on. But he didn't give much of example.

  Hopefully the example below will help people incorporate such functionality in our apps. Probably with a little work we could write a class to replicate most of the browser functionality. At least all that can be replicated via JavaScript. And just include it in our apps.

  http://thesaj.wordpress.com/2009/12/05/how-to-enable-cntrl-t-tab-in-flash/

Is there a way to set a asp.net button's CommandArgument in javascript?

I have a GridView that lists a bunch of items and one of the columns has a link that displays a modal (AjaxToolkit ModalPopupExtender). Let's call that link "Show". In that modal, I have a asp:button for saving the data entered in that modal. Let's call that button "Save"

So when the user clicks on a "Show" link in a certain row, I'd like write some javascript that sets something in the "Save" button, so that in my code-behind, I can handle "Save".Command and use the CommandEventArgs parameter to get the value.

Is this possible, or do I just need to use a hidden input tag and set its value?

From stackoverflow

• Not a direct answer to your question, but another possible way of solving the problem:

  Place a HiddenField control on the page. In your code-behind, before displaying the modal popup, set the value of that control to the ID of the row that was clicked (or the row number, or some identifying value). Then in the code-behind of your Save button, you can just read the value of the HiddenField.

  slolife : Thanks Brant. Looks like that is the workaround.

• Well, after continuing the research, it looks like it cannot be done. The CommandArgument property might reside in the ViewState, but for this case, it is completely server side and cannot be changed using javascript.

Overriding int on a bit enum

I saw some .NET 2.0 code which looked like this:

 public enum abc : int
 {
  value1 = 1,
  value2 = 2,
  value3 = 4
 }

etc...

Now I know the flags enum can be used for bit values (like above, right?) so you can do | "or" &, etc, but what is the reason for not using that? What is the point in overriding int?

If I design an enum for choosing multiple values I would use the flags attribute and not override anything (what I don't understand). What does the above approach net?

Thanks

From stackoverflow

• The ": int" is specifying what to use as the underlying type for enum values; any integral type other than char can be used here. The default is int, so it is redundant in this case.

• It's not "overriding" int, it's indicating what type the enumeration is based on. The default is int so this actually isn't achieving anything, but in some cases you might want to base an enum on long to store more flags, or byte to save space if there will be very many instances of the enum around and it only has a few values, however the recommendation in the framework design guidelines is to use int unless you have a very good reason not to.

  If this is intended to be a flags enumeration then it should be marked with [Flags], and this appears to be the intention here, so the omission looks like a mistake. You can still combine the values using the bitwise operators even if it isn't marked with [Flags], but from memory I think you get a compiler warning (or it might be an FxCop warning, or possibly a ReSharper warning...).

• I don't see any benefit. Probably lack of experience on the programmer's part.

• FlagsAttribute only tells the user that fields in this enum can be combined; it doesn't actually set the fields of the enum to "flaggable" values. This, you will have to do yourself, just like you have already.

• If you are asking the scenario for choosing specific values, I don't know a good example offhand, but I can imagine something like

  enum Color = {
    Red = 0xFF0000,
    Green = 0x00FF00,
    Blue = 0x0000FF }
  

  maybe being useful.

• Setting the values on a non-Flags enum to use a 1-hot encoding just for the sake of using a 1-hot encoding doesn't really make any sense. I guess it might make sense if you thought you might make it a Flags enum in the future. Otherwise, the values should either be not overridden or assigned a value based on their meaning (eg. Color.Red = 0xFF0000).

• In reference to the ": int" specifying the underlying type, if I saw that, I'd have a look around and see if they delcare all their enums that way. If so, they probably just learnt to write enums that way ( or maybe they are a hyper-explicit coder). If however, just this enum was declared this way, perhaps the programmer is trying to say something (that might have been better said with a comment) that it is critical that this has an underlying "int" type. Perhaps it is being binary serialised or deserialised somewhere.

  As for the [Flags] attribute, as far as C# is concerned, you can use bitwise operators on any enum, not just those declared with [Flags]. From your above example, without any [Flags], the following is totally legal:

  abc myAbcValue = value1 | value2;

  The primary difference with respect to the [Flags] is in the ToString() and Parse() methods of Enum. Without [Flags], myAbvValue.ToString() will return "3", but with [Flags] it will return "value1, value2". Likewise, with the [Flags],

  abc yourAbcValue = (abc)Enum.Parse( typeof(abc), "value1, value2") ;

  will set the underlying value of yourAbcValue to 3.

  Note that the IsDefined() method of Enum() does not allow for multiple values with [Flags] enums, so:

  Enum.IsDefined( typeof(abc), "value1, value2")

  returns false regardless of the use of [Flags].

  Also note that [Flags] provides information for any reflection access, and other languages may treat the restriction on flagged enums differntly to normal enums.

How to make a structure map powered viewengine in asp.net mvc

My views extend a base view class ive made:

public class BaseView : ViewPage

At the moment im calling ObjectFactory.GetInstance inside this class' constructor to get some interface implementations but id like to use structuremap to inject them as constructor arguments.

Im using a structuremapcontrollerfactory to create my controllers, but how can i do the same for views?

I know i can implement a custom ViewEngine, but using reflector to look at the mvc default viewengine and its dependencies, it seems to go on and on and i'd rather not have to re-implement stuff thats already there. Has anyone got a cunning idea how to solve this?

I know i could make things easier with setter instead of constructor injection but id rather avoid that if possible.

From stackoverflow

• Yes, if you are talking about the WebForms ViewEngine, you are correct in your assessment that it was not designed with inversion of control in mind. You can either lobby Microsoft to change it, or you can use setter injection. In FubuMVC, we use setter injection along with StructureMap's BuildUp method:

  • WebFormsControlBuilder creates the instance of the view
  • StructureMapWebFormsControlBuilder derives from WebFormsControlBuilder - calls the base to create the instance, and then calls BuildUp to perform setter injection on all applicable properites.
  Andrew Bullock : setter injection it is

@“Data Source=(local)\ SIRSYSTEM \ HEY;” – can’t connect if a string includes SIRSYSTEM

Hi

The name of my MS Sql server instance is HEY, but when trying to log on to Sql server via Microsoft Sql management studio, the login window displays as the name of a server "SIRSYSTEM\HEY".

What is SIRSYSTEM? A name of default server instance? If so, then why can’t I connect to HEY using the following connection string ( in Ado.net ):

@"Data Source=(local)\ SIRSYSTEM \ HEY;Initial Catalog=FilmClub;"+ "Integrated Security=true";

cheers

From stackoverflow

• SIRSYSTEM is not a system name but likely something that you or someone typed at some point as the default or is it the name of the server itself?

  Data Source=(local)\SIRSYSTEM\HEY is not at all valid. Try just (local)\HEY.

  You might have to unregister and then reregister the server instance within management studio to change this.

• All you need to do is use 'Hey', SIRSYSTEM is the name of the computer.

  Or: localhost\HEY

• SIRSYSTEM is the server or workstation name hosting the instance; to connect to, it the datasource would be either .\HEY, or SIRSYSTEM\HEY, get rid of the (local)\

C++ - How to set file permissions (cross platform)

Hi guys. I am using C++ ofstream to write out a file. I want to set the permissions to be only accessible by the user: 700. In unix; I suppose I can just issue a system("chmod 700 file.txt"); but I need this code to work on windows as well. I can use some windows api; but what is the best c++ cross platform way to do this?

From stackoverflow

• There is no cross-platform way to do this. Windows does not support Unix-style file permissions. In order to do what you want, you'll have to look into creating an access control list for that file, which will let you explicitly define access permissions for users and groups.

  An alternative might be to create the file in a directory whose security settings have already been set to exclude everyone but the user.

• No idea if it would work, but you could look into using the chmod.exe executable that comes with Cygwin.

  rmeador : +1 because it's an interesting idea. I know that chmod under cygwin does The Right Thing every time I've ever tried it.

  aib : What? It changes NTFS ACLs?

• There's no standard way to do this in C++, but for this special requirement you should probably just write a custom wrapper, with #ifdef _WIN32. Qt has a permission wrapper in it's QFile class, but this would of course mean depending on Qt ...

  Ferruccio : The warning on the link you provided implies that this might not work as expected on Windows.

• You can't do it in a cross-platform manner. In Linux, you should use the function chmod(2) instead of using system(2) to spawn a new shell. On Windows, you'll have to use the various authorization functions to make an ACL (access-control list) with the proper permissions.

• The system() call is a strange beast. I have been bitten by a NOP system() implementation on a Mac many moons ago. It's implementation defined meaning the standard doesn't define what an implementation (platform/compiler) is supposed to do. Unfortunately, this is also about the only standard way of doing something outside the scope of your function (in your case -- changing the permissions).

  Update: A proposed hack:

  • Create a non-empty file with appropriate permissions on your system.

  • Use Boost Filesystem's copy_file to copy this file out to your desired output.

    void copy_file(const path& frompath, const path& topath): The contents and attributes of the file referred to by frompath is copied to the file referred to by topath. This routine expects a destination file to be absent; if the destination file is present, it throws an exception. This, therefore, is not equivalent to the system specified cp command in UNIX. It is also expected that the frompath variable would refer to a proper regular file. Consider this example: frompath refers to a symbolic link /tmp/file1, which in turn refers to a file /tmp/file2; topath is, say, /tmp/file3. In this situation, copy_file will fail. This is yet another difference that this API sports compared to the cp command.

  • Now, overwrite the output with actual contents.

  But, this is only a hack I thought of long after midnight. Take it with a pinch of salt and try this out :)

• Ironically, I have just run into this very same need earlier today.

  In my case, the answer came down to what level of permission granularity I need on Windows, versus Linux. In my case, I only care about User, Group, and Other permission on Linux. On Windows, the basic Read/Write All permission leftover from DOS is good enough for me, i.e. I don't need to deal with ACL on Windows.

  Generally speaking, Windows has two privilege models: the basic DOS model and the newer access control model. Under the DOS model there is one type of privilege: write privilege. All files can be read, so there is no way to turn of read permission (because it doesn't exist). There is also no concept of execute permission. If a file can be read (answer is yes) and it is binary, then it can be executed; otherwise it can't.

  The basic DOS model is sufficient for most Windows environments, i.e. environments where the system is used by a single user in a physical location that can be considered relatively secure. The access control model more complex by several orders of magnitude.

  The access control model uses access control lists (ACL) to grant privileges. Privileges can only be granted by a process with the necessary privileges. This model not only allows the control of User, Group, and Other with Read, Write, and Execute permission, but it also allows control of files over the network and between Windows domains. (You can also get this level of insanity on Unix systems with PAM.)

  Note: The Access Control model is only available on NTFS partitions, if you are using FAT partitions you are SOL.

  Using ACL is a big pain in the ass. It is not a trivial undertaking and it will require you to learn not just ACL but also all about Security Descriptors, Access Tokens, and a whole lot of other advanced Windows security concepts.

  Fortunately for me, for my current needs, I don't need the true security that the access control model provides. I can get by with basically pretending to set permissions on Windows, as long as I really set permissions on Linux.

  Windows supports what they call an "ISO C++ conformant" version of chmod(2). This API is called _chmod, and it is similar to chmod(2), but more limited and not type or name compatible (of course). Windows also has a deprecated chmod, so you can't simply add chmod to Windows and use the straight chmod(2) on Linux.

  I wrote the following:

  #include <sys/stat.h>
  #include <sys/types.h>
  
  #ifdef _WIN32
  #   include <io.h>
  
  typedef int mode_t;
  
  /// @Note If STRICT_UGO_PERMISSIONS is not defined, then setting Read for any
  ///       of User, Group, or Other will set Read for User and setting Write
  ///       will set Write for User.  Otherwise, Read and Write for Group and
  ///       Other are ignored.
  ///
  /// @Note For the POSIX modes that do not have a Windows equivalent, the modes
  ///       defined here use the POSIX values left shifted 16 bits.
  
  static const mode_t S_ISUID      = 0x08000000;           ///< does nothing
  static const mode_t S_ISGID      = 0x04000000;           ///< does nothing
  static const mode_t S_ISVTX      = 0x02000000;           ///< does nothing
  static const mode_t S_IRUSR      = mode_t(_S_IREAD);     ///< read by user
  static const mode_t S_IWUSR      = mode_t(_S_IWRITE);    ///< write by user
  static const mode_t S_IXUSR      = 0x00400000;           ///< does nothing
  #   ifndef STRICT_UGO_PERMISSIONS
  static const mode_t S_IRGRP      = mode_t(_S_IREAD);     ///< read by *USER*
  static const mode_t S_IWGRP      = mode_t(_S_IWRITE);    ///< write by *USER*
  static const mode_t S_IXGRP      = 0x00080000;           ///< does nothing
  static const mode_t S_IROTH      = mode_t(_S_IREAD);     ///< read by *USER*
  static const mode_t S_IWOTH      = mode_t(_S_IWRITE);    ///< write by *USER*
  static const mode_t S_IXOTH      = 0x00010000;           ///< does nothing
  #   else
  static const mode_t S_IRGRP      = 0x00200000;           ///< does nothing
  static const mode_t S_IWGRP      = 0x00100000;           ///< does nothing
  static const mode_t S_IXGRP      = 0x00080000;           ///< does nothing
  static const mode_t S_IROTH      = 0x00040000;           ///< does nothing
  static const mode_t S_IWOTH      = 0x00020000;           ///< does nothing
  static const mode_t S_IXOTH      = 0x00010000;           ///< does nothing
  #   endif
  static const mode_t MS_MODE_MASK = 0x0000ffff;           ///< low word
  
  static inline int my_chmod(const char * path, mode_t mode)
  {
      int result = _chmod(path, (mode & MS_MODE_MASK));
  
      if (result != 0)
      {
       result = errno;
      }
  
      return (result);
  }
  #else
  static inline int my_chmod(const char * path, mode_t mode)
  {
      int result = chmod(path, mode);
  
      if (result != 0)
      {
       result = errno;
      }
  
      return (result);
  }
  #endif
  

  It's important to remember that my solution only provides DOS type security. This is also known as no security, but it is the amount of security that most apps give you on Windows.

  Also, under my solution, if you don't define STRICT_UGO_PERMISSIONS, when you give a permission to group or other (or remove it for that matter), you are really changing the owner. If you didn't want to do that, but you still didn't need full Windows ACL permissions, just define STRICT_UGO_PERMISSIONS.

• I just found a couple of ways to do chmod 700 easily from the Windows command line. I'm going to post another question asking for help coming up with an equivalent win32 security descriptor structure to use (if I can't figure it out in the next few hours).

  Windows 2000 & XP (messy- it always seems to prompt):

  echo Y|cacls *onlyme.txt* /g %username%:F
  

  Windows 2003+:

  icacls *onlyme.txt* /inheritance:r /grant %username%:r
  

  EDIT:

  If you had the ability to use the ATL, this article covers it (I don't have Visual Studio available): http://www.codeproject.com/KB/winsdk/accessctrl2.aspx

  Actually, it says the sample code includes non-ATL sample code as well- it should have something that works for you (and me!)

  The important thing to remember is to get r/w/x for owner only on win32, you just have to wipe all of the security descriptors from the file and add one for yourself with full control.

Does a new or emerging programming language really need practical libraries?

Using a factor or forth, Arc or whatever as an example (note: factor is a bad example because it has a large set of practical libraries). Lets say you are considering using a programming language. Does having a large set of practical libraries matter? If your language is well designed, then it would be easy to create a 'string' library or a 'date' library. Maybe even a web framework?

I mention this, because when a language emerges, it seems that someone brings up 'practical libraries'.

From stackoverflow

• Practical libraries are important. I don't get paid to write a framework, I get paid to add value to a business. If I told a client I had to charge them to write a string data type, I'd probally lose my job / contract.

• It definitely matters - nine times out of ten, I'm going to pick a language with good, well-supported libraries that help accomplish what I want to do.

  This isn't to say I wouldn't enjoy writing my own libraries (quite the contrary), but for a production project where quick, accurate results are important, that wouldn't be a practical option by any stretch of the imagination.

• Just look at the majority of questions here on SO to see how important tools are to people.

  That said, if you find a new language that fits your task really well and you have the time and inclination to write your own tools then by all means dive in. That's one way that new languages get nice libraries, after all.

• Java and .Net have spoiled people with the abundance of classes in the framework or in additional high-quality libraries (quite often free and open source as well). Same goes for Ruby and Python. It'd be hard to adopt a new language without such a library, as your productivity will suffer teremndously by having to reimplement every single feature you need.

  Unless it's a breakthrough language that introduces something radical like intentional programming (I tell the computer what I want it to do and it inferes the proper code to do it)... Why, you have one of those? :-)

• A large library makes a language much more productive to use. A really great language without support for parsing XML, crypto, a web framework, a UI framework, etc. takes a lot more time to produce working code with. For learning purposes, a language without a large library is fine, but for practical purposes, it is going to cost time and money to use such a language. Imagine if every time you wanted to load an image you had to write code to parse the .jpg headers. What if you had to hand-code your XML parser rather than loading it into a pre-built one. You'd likely screw it up and spend a lot of time debugging. If the goal of the project is to create a new tool, writing support code is not a great use of your time.

• I think that the big reason for Python's popularity is its huge standard library. Same goes with Java and PHP. In fact, I'd probably say that the selection of libraries is more important than the language itself.

  TokenMacGuy : I'd go so far as to say that the ONLY thing going for PHP (and possibly java) is the library they come with. as languages go, they're both pretty dreadful.

  Jason Baker : I agree. And that's why I don't do as much C# programming as I'd like. There simply isn't a lot of high-quality open source software for C#.

• If your top priority is to create a finished product with the least amount of time and effort possible, then yes, having available libraries matters. (If your goal is fun, or learning for the sake of learning, then writing your own libraries can be a good experience.)

  A good library is mature enough that lots of people have used it and weeded out most of the bugs. It doesn't matter how great your programming language is or how easy it is to write a library from scratch. There's no replacement for having your code tested over time.

  A lot of libraries are not interesting or fun to write, and reimplementing them again is not going to revolutionize the world in any way. There is only so much you can do with a date library or a string library whatever. It either works or it doesn't. Many libraries are simply an implementation of a standard or of some de facto standard behavior, and someone just has to slog through the necessary work to get it right. The less of this you have to do yourself, the better.

  Any brand new language that can take advantage of existing libraries starts off way ahead, in my opinion. Clojure for example, though a very new language, also has access to all the libraries of Java. This is arguably one big reason it's doing so well at the moment. Effort is put toward novel things rather than re-inventing wheels.

• It's a tempting idea, and it certainly works if you're Paul Graham or Chuck Moore.

  It might work if your domain is very very bounded, and you're not going to get a requirement thrown at you that's outside that domain, something as simple as a client asking for an "import from Excel" feature. On the other hand, Paul Graham used Lisp to write a web shop system, which is a very broad domain of requirements; I'd be interested in knowing how he handled something like a PDF export, would he have given the PDF spec and a Lisp manual to some intern on summer vacation from MIT, or would he have gone down to the C libraries?

  It might work if your domain is driven by logic or by enduring natural principles, something like an astronomy simulation. If they're human requirements, they'll be full of contradictions and special cases (string and date libraries fall into that category, by the way), and there is no abstraction or language feature that entirely cuts through that, you'll have to slog through the special cases whether you're writing in Haskell or PHP.

  It might work where optimisation is very very important (EDIT: and where you're smart enough to optimise it yourself) - you have a stripped down system where you know every layer of the stack because you've implemented it yourself with a particular goal in mind.

  I associate the whole cluster of ideas with grad students: they're in the top 1% in programming skills and general smartness; they're working in a very narrow domain; they may not have the best equipment so they're trying to strip things down and optimise in depth; and they don't have the learning vs getting work done dilemma of working programmers.

• You think string libraries are easy to write? Go have a look at Unicode, UTF-8, UTF-16, legacy codepages, byte order issues and whatnot.

  You think date/time libraries are easy to write? Go have a look at leap seconds, week numbering schemes and whatnot.

  Having these kinds of things thought out for you, implemented once and implemented properly, saves more time and headaches than you think.

How to integrate my Web Application with Skype, Gtalk of another IM system?

I want to be able to send reminders to my users, using their IM system of choice, very much like Remember The Milk does when it sends notifications.

How can I do that?

Additional INFO: I develop in ASP.Net, SQL Server 2008 and Windows 2008

From stackoverflow

• These may be overkill for your problem but these servers implement the Jabber (Gtalk etc) protocol and have add-ins which support sending messages over all the major IM networks.

  http://www.process-one.net/en/ejabberd/

  http://www.igniterealtime.org/projects/openfire/

  http://jabberd.org/

  You might be able to integrate one of them into your solution to solve your IM needs.

• gtalk is Jabber. Look for Jabber bots, there are some that will work in your environment. For example one in Python.

  Skype requires instance of Skype running on your servers, and your application talking to it via API.

  Each of the systems require its own approach. Thus your mileage can vary.

• Just as an FYI, the GTalk servers also require Transport Level Security (TLS) so make sure to look for that in any library you are considering for your XMPP communications.

• As mentioned above you could use your own jabber server create an account on that server and then add a gateway for each IM protocol you wish to support. This way, you use a jabber api to send messages on one account (your local jabber account) and then the server will pass through the gateways to the various protocols.

  Another way to do this would be to use libpurple. This is the library used to power the multi protocol support in pidgin and adium. This way would have less setup but maybe higher learning curve.

In c# is it possible for a function to only be called from within another function?

In c# is it possible to create a function that can only be called from within another function?

eg can you do something like this?

private void a() {

b(); c(); ...do something else

private void b() { ..do something but can only be called from a() }

private void c() { ..do something but can only be called from a() }

}

The reason I want to do this is that function b() and c() split some implentation details of a() and they are just cleaner and easier to read in their own scope. However, these functions are of no use to the class as a() does some handling after they are called which must take place.

From stackoverflow

• Not exactly but you could implement both within their own class. Mark b() as private.

• Use an anonymous nested function maybe?

  Richard : Pendantic: You mean delegate of course.

  apphacker : Yes, a delegate.

  Jason Slocomb : Pedantic: You mean pedantic of course.

  Cheeso : What good developer is not a pedant?

  plinth : Pedants rule, OK. Or more precisely, exhibit certain of the conventional trappings of leadership

• The short answer is no; however, you can create an anonymous delegate or lambda expression as your internal b() method.

  Cheeso : Yes, and See code above (by Josh).

  Ryan Riley : Yes, Josh's code is also what I noted above; however, that's not the same as the sample in the original question, which appears to come from a JavaScript example. Josh's example, though, is perfect.

• You could use the internal keyword and put both those functions inside the same class, while leaving other other functions in a different class: http://msdn.microsoft.com/en-us/library/7c5ka91b.aspx

  Dave : An internal method is more accessible than a private method...

• Using a delegate you can do:

  public voidMyFunction()
  {
  
     Func<string> myFunction=(s)=>Console.WriteLine(s);
  
     foreach(string str in myStringList)
     {
        myFunction(str);
     }
  }
  

  Andrew Robinson : Better than my solution. Nice example.

  Cheeso : This is the best way to do it.

  Jon Erickson : delegates/lambdas are the way to go.

• To gain the effect of only a() calling b(), either do as Andrew noted already, by putting a() and b() in a class and marking b() appropriately. If you're working inside of an assembly that you control totally, you could use internal instead of private if a() and b() will be in different classes, but in the same assembly. Then user code cannot call it (from outside of your assembly, that is, from their application program) and you can control via policy the writing of your assembly.

  Cheeso : The problem with internal and private and other such modifiers - they work during a regular compile, but they can be circumscribed via reflection. I can, with reflection, read and even modify a private field on a class.

  Michael Trausch : Then you can probably do it with an anon method, too; that gets compiled into its own method. Try looking at the disassembly (CLI) of a small example class w/ an anon method. It compiles to a private method (static or not depending on context). i.e.: http://zest.spicerack.trausch.us/~mbt/csharp-ex/

  Cheeso : BTW, here's an example of using Reflection to tickle private fields: http://cheeso.members.winisp.net/srcview.aspx?file=PrivateReflection2.cs

• You can also create something like this:

  internal abstract class SecretFunctionWrapper
  {
      private void MySecretFunction()
      {
          ...
      }
  
      protected void FunctionWhichCalls()
      {
          ...
          MySecretFunction();
      }
  }
  
  public MyRealClass : SecretFunctionWrapper
  {
      ...
  }
  

  This will work only for one function. You can also try nested private class like this:

  public class A
  {
      private static class Wrapped
      {
          private static void A()
          {
              secred code
          }
  
          public static void B()
          {
              A();
          }
      }
  
      public void UsingA()
      {
          Wrapped.B();
      }
  }
  

• Well you could use reflection and just get the calling method name and throw an exception if it were anything other than A.

  http://www.csharp-examples.net/reflection-calling-method-name/

  But if b and c are private they can only be called from within that class anyway, and if you're the only one that is writing the class, then i fail to see the problem. So it seems to me its not a coding problem but rather one of policy.

  I'd just document the intent in the method headers/comments.

  Similar Question Here - Note the comments on the answer

  Simon Buchan : b() and c() as nested delegates can access a()'s locals.

• i dont know but maybe Code by Contracts may help but this is not supported natively

• You could use the StackFrame class to check at runtime who's the caller of the function:

  public class MyClass
  {
      public static void A()
      {
         B();
      }
  
      public static void B()
      {
          var stackTrace = new StackTrace();
  
          if (stackTrace.FrameCount < 1 || stackTrace.GetFrame(1).GetMethod() != typeof(MyClass).GetMethod("A"))
                throw new InvalidOperationException("Not called from A()");
      }
  }
  

  But that is

  1) Only at runtime

  2) Slow

  3) A really dirty hack

  Wim Coenen : also won't work when using an obfuscator or when parts of the callstack are optimized away by the runtime

  plinth : Also, stack walking has some rather unpalatable differences in 64 bit.

  JoshBerke : Stack walking in release mode can result in differences when functions are inlined.

• I wouldn't worry about taking explicit steps to ensure b() and c() are only called by a().

  It makes sense to worry about the public methods you expose on a class, since you're providing an interface to the outside world, potentially to people who don't have access to the source code of your class (or at the very least don't want to worry about the implementation details of your class).

  Inside your class, though, you should feel free to have whatever private methods you want for whatever reasons you want. Code re-use is one reason to create a new private method, but creating multiple smaller, single-use methods to break up a larger one is also a perfectly valid (and common) reason.

  Beyond that, for future maintainers of your code a simple comment like:

  //this method should only be called by a()
  private void b()
  {
      ...
  }
  

  is going to be far more understandable than most of the other solutions presented here.

  Zemm : +1 because this makes code that is much more readable than the delegate/lambda answers. Although I'd use the /// auto doc comment on b() and c() so that the warnings would show up in the IntelliSense pop ups.

  Simon Buchan : I would really hope maintainers of my code find inner functions easy... wishful thinking?

  John Price : @Simon it's not so much what's "easy" as what's "expected". In the straightforward case of just wanting to chop up a big method, why not stick with convention and use private methods?

  Randolpho : Although an anonymous method / delegate answers the original question, I agree that your answer is the better practice. +1.

• Maybe easier to use #region in this case

Parsing an HTML file with selectorgadget.com

How can I use beautiful soup and selectorgadget to scrape a website. For example I have a website - (a newegg product) and I would like my script to return all of the specifications of that product (click on SPECIFICATIONS) by this I mean - Intel, Desktop, ......, 2.4GHz, 1066Mhz, ...... , 3 years limited.

After using selectorgadget I get the string- .desc

How do I use this?

Thanks :)

From stackoverflow

• Inspecting the page, I can see that the specifications are placed in a div with the ID pcraSpecs:

  <div id="pcraSpecs">
    <script type="text/javascript">...</script>
    <TABLE cellpadding="0" cellspacing="0" class="specification">
      <TR>
        <TD colspan="2" class="title">Model</TD>
      </TR>
      <TR>
        <TD class="name">Brand</TD>
        <TD class="desc"><script type="text/javascript">document.write(neg_specification_newline('Intel'));</script></TD>
      </TR>
      <TR>
        <TD class="name">Processors Type</TD>
        <TD class="desc"><script type="text/javascript">document.write(neg_specification_newline('Desktop'));</script></TD>    
      </TR>
      ...
    </TABLE>
  </div>
  

  desc is the class of the table cells.

  What you want to do is to extract the contents of this table.

  soup.find(id="pcraSpecs").findAll("td") should get you started.

• Have you tried using Feedity - http://feedity.com for creating a custom RSS feed from any webpage.

Downloading files

Hello,

I am using asp.net/C#. I have a url to a folder on a remote server. In that folder are images that are all .jpgs. I don't know the names of the files, just that they are all .jpgs. How do I download all the .jpg images using asp.net/C# to a folder on my local hard drive.

Just to clarify. I am pulling files from a remote server and I am saving them to my local machine. I was given a web URL and told that the files I needed to pull down every night where .jpg image files. That's all I was told. I have no idea how I can get a list of files on a remote server with just the url to the folder.

Thanks

From stackoverflow

• If it's a web URL, you'd have to depend on the web server giving you some sort of list of files. The format of the list could be almost anything.

  Put it this way: using a browser or anything else, how would you as a human find out all the filenames?

  Just to clarify, are you writing code on the server which has the files? If so, you can find out what files are present using Directory.GetFiles. What do you want the user to have to do at the local side?

  If you could make your question a bit clearer it would really help.

• Here is some concept code to work with

  DirectoryInfo di = new DirectoryInfo("M:\MappedDrive");
   FileInfo[] rgFiles = di.GetFiles("*.aspx");
   foreach(FileInfo fi in rgFiles)
   {
    Response.Write("<br><a href=" + fi.Name + ">" + fi.Name + "</a>");       
   }
  

While installing redbox plugin to rails why does it say Plugin not found ?

When I try to install redbox plugin although everything gets installed, a last line says plugin not found. Why is that ?

script/plugin install http://svn2.cvsdude.com/craigambrose/plugins/redbox
+ ./README
+ ./Rakefile
+ ./images/redbox_spinner.gif
+ ./init.rb
+ ./install.rb
+ ./javascripts/redbox.js
+ ./lib/redbox.rb
+ ./lib/redbox_helper.rb
+ ./stylesheets/redbox.css
+ ./tasks/redbox_tasks.rake
+ ./test/redbox_test.rb
Plugin not found: ["http://svn2.cvsdude.com/craigambrose/plugins/redbox"]

From stackoverflow

• The paths inside vendor/plugins/redbox/install.rb appear to be wrong:

  plugins_dir = File.expand_path(".")
  redbox_dir = File.join(plugins_dir, 'redbox')
  root_dir = File.join(redbox_dir, '..', '..', '..')
  
  File.copy File.join(redbox_dir, 'javascripts', 'redbox.js'), File.join(root_dir, 'public', 'javascripts', 'redbox.js')
  File.copy File.join(redbox_dir, 'stylesheets', 'redbox.css'), File.join(root_dir, 'public', 'stylesheets', 'redbox.css')
  File.copy File.join(redbox_dir, 'images', 'redbox_spinner.gif'), File.join(root_dir, 'public', 'images', 'redbox_spinner.gif')
  

  The redbox_dir path isn't accounting for the vendor/plugins path components. But you can complete the installation yourself by manually copying the three files listed above from the vendor/plugins/redbox directory to the appropriate subdirectories beneath your app's public directory.

  Ram : Is this the correct approach ? I mean do I need to do this with all plugins or is this a problem with redbox ?

  jcrossley3 : It's only a problem with redbox, but in general, any time you see "Plugin not found", check the plugin's install.rb file to see if you can determine the "real" error.

  Ram : it works fine if the pwd is changed to vendor/plugin

How to copy between generic declared types with value type constraints

I have a generic method that copies values between value types. The following approaches give a design time error, even with the struct constraint. Any idea how I can copy or cast between the values?

    private Ttgt MyMethod<Tsrc,Ttgt>(Tsrc SourceObject) 
        where Tsrc:struct
        where Ttgt:struct
    {
        //Error:cannot implictly convert type 'Tsrc' to 'Ttgt'
        Ttgt returnObject = SourceObject; 

        //Error:Cannot convert type 'Tsrc' to 'Ttgt'
        Ttgt returnObject = (Ttgt)SourceObject; 

        return returnObject;
    }

From stackoverflow

• //Error:Cannot convert type 'Tsrc' to 'Ttgt'

  You cannot convert between arbitrary types, unless there is an accessible conversion operator.

• The two are defined as different types. Even though they are both structs, they are not the same types.

  Define both souce and target as the same type:

  private T MyMethod<T>(T source, T target)
  {
  

• Given that there is a registered type converter for the types that you're trying to convert between a little reflection magic could do the trick:

  private Ttgt MyMethod<Tsrc,Ttgt>(Tsrc sourceObject) 
      where Tsrc:struct where  Ttgt:struct    
  {    
      Type targetType = typeof(Ttgt);
      TypeConverter tc = TypeDescriptor.GetConverter(targetType);
      Ttgt returnObject = (Ttgt)tc.ConvertTo(sourceObject, targetType);
      return returnObject;    
  }
  

  But out of the box it would be of very limited use since there is no converter between bool and int for example. What problem are you trying to solve?

  I also discovered another question with some crazy conversion code in it.

  Edit: Your comment makes it clear that you are trying to perform object to object mapping between domain objects and some kind of view/contract model. Have you looked at AutoMapper?

  Jon Simpson : I am writing a recursive object-to-object mapper. to seperate the contract classes from the underlying business logic classes in a WCF implementation. The implementation classes are near identical to the contract but I don't want to introduce a dependency between the two.

  mike nelson : This should say: Type targetType = typeof(Ttgt);

  PHeiberg : Thanks. Updated variable names to be correct.

• This is "By Design." You are attempting to cast between two unrelated value types. This will never succeed and therefore it is flagged as an error.

  This will be true of all value typse because they are implicitly sealed. In order for a cast between TSrc -> Ttgt to succeed, there must be a class hierarchy relationship between the two types. This cannot be because all value types are sealed hence there is no way one can derive from the other.

  The only way this could succeed is if one had a custom conversion operator between the types. This may be the case. However when dealing with generic types, custom conversion operators will not be processed.

• The Convert class exists for this exact purpose.

  private Ttgt MyMethod<Tsrc, Ttgt>(Tsrc SourceObject)
      where Tsrc : struct
      where Ttgt : struct
  {
      return (Ttgt) Convert.ChangeType(SourceObject, typeof(Ttgt));
  }
  

  Also note that you could do this:

      return (Ttgt) (object) SourceObject;