Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Critical Critical
    • Resolution: Unresolved
    • Affects Version/s: 3.1.2_b23
    • Fix Version/s: None
    • Component/s: distributed management
    • Labels:
      None
    • Environment:

      Windows server 2008 R2 Sp1
      Glassfish 3.1.2 Release

      Description

      I initially tried using a windows domain account to install the node and that didn't work as per GLASSFISH-18327. That issue was also incorrectly marked as resolved. Network captures show that the release version of Glassfish 3.1.2 still does not use the domain account, but attempts to use the local account.

      After giving up with this, I created a new local account on the remote machine called glassfish. Granted full access to the 2 required registry keys and added the account to the administrators group. Attempting to install the remote node using this account still fails with the following message:

      Successfully verified that the host, hostname, is not the local machine as required. Successfully resolved host name to: hostname/10.65.30.xxx Successfully connected to DCOM Port at port 135 on host hostname. Successfully connected to NetBIOS Session Service at port 139 on host hostname. Successfully connected to Windows Shares at port 445 on host hostname. The remote file, C: doesn't exist on hostname: Access is denied.

      I performed a network capture and can tell you the following:

      1. The user account is successfully authenticated with STATUS_SUCCESS (0x00000000)
      2. SMB is attempting to access \\hostname\C$ no matter what I set the remote test directory to.
      3. NT Status: STATUS_FS_DRIVER_REQUIRED (0xc000019c) is returned from the remote host but I suspect this is normal and used for dynamic library loading for the file system.

      4. NT Status: STATUS_ACCESS_DENIED (0xc0000022) is returned on attempting to connect to \\hostname\C$

      The documentation does not state any other prerequisite or permissions that need to be setup for this to function. What is missing?

        Activity

        Hide
        Byron Nevins added a comment -

        what is the exact command you're running?

        Show
        Byron Nevins added a comment - what is the exact command you're running?
        Hide
        jp2011 added a comment -

        To make things even simpler, it is reproducible by the validate-dcom command alone.

        Password file contains the following line: AS_ADMIN_WINDOWSPASSWORD=$

        {ALIAS=glassfish-alias}

        I have setup the alias already in asadmin as per the documentation.

        c:\glassfish3\bin>asadmin --passwordfile passwordfile.txt validate-dcom -w glassfish remotehost
        remote failure:
        Successfully verified that the host, remotehost, is not the local machine as required.
        Successfully resolved host name to: remotehost/10.65.30.187
        Successfully connected to DCOM Port at port 135 on host remotehost.
        Successfully connected to NetBIOS Session Service at port 139 on host remotehost
        nc.
        Successfully connected to Windows Shares at port 445 on host remotehost.
        The remote file, C: doesn't exist on remotehost: Access is denied.

        Command validate-dcom failed.

        I can speak to the network capture I took as well, but that would be easier offline to this web portal.

        Show
        jp2011 added a comment - To make things even simpler, it is reproducible by the validate-dcom command alone. Password file contains the following line: AS_ADMIN_WINDOWSPASSWORD=$ {ALIAS=glassfish-alias} I have setup the alias already in asadmin as per the documentation. c:\glassfish3\bin>asadmin --passwordfile passwordfile.txt validate-dcom -w glassfish remotehost remote failure: Successfully verified that the host, remotehost, is not the local machine as required. Successfully resolved host name to: remotehost/10.65.30.187 Successfully connected to DCOM Port at port 135 on host remotehost. Successfully connected to NetBIOS Session Service at port 139 on host remotehost nc. Successfully connected to Windows Shares at port 445 on host remotehost. The remote file, C: doesn't exist on remotehost: Access is denied. Command validate-dcom failed. I can speak to the network capture I took as well, but that would be easier offline to this web portal.
        Hide
        Byron Nevins added a comment -

        Can you access the c$ share from another computer – say

        net use X: \\other\c$

        ?

        Show
        Byron Nevins added a comment - Can you access the c$ share from another computer – say net use X: \\other\c$ ?
        Hide
        Byron Nevins added a comment -

        Please make sure theses items are setup correctly, especially the third one:

        1. Server service is in the started state and is set to start automatically.
        2. Remote Registry service is also in the started state and is set to start automatically.
        3. Set the Local Policy for Network Access:Control Panel" > "Administrative Tools" -> "Local Security Policy"> "Local Policies" -> "Security Options" -> "Network Access: Sharing security model for local accounts" Make sure it is set to Classic

        Show
        Byron Nevins added a comment - Please make sure theses items are setup correctly, especially the third one: 1. Server service is in the started state and is set to start automatically. 2. Remote Registry service is also in the started state and is set to start automatically. 3. Set the Local Policy for Network Access:Control Panel" > "Administrative Tools" -> "Local Security Policy" > "Local Policies" -> "Security Options" -> "Network Access: Sharing security model for local accounts" Make sure it is set to Classic
        Hide
        ljnelson added a comment - - edited

        I have exactly the same problem.

        I installed and ran setup-local-dcom on the remote machine as an administrator. It claimed it ran successfully.

        Then I made sure that your steps 1-3 above were taken. I had to manually start the remote registry service.

        My remote machine is running Windows 7 Professional on a 64-bit machine with all updates installed.

        Here is my command and output:

        ljnelson$ asadmin --passwordfile ~/.glassfish.passwords --port=9048 validate-dcom --windowsuser lnelson --windowsdomain jenzabar --remotetestdir 'C:\crap' --verbose true 10.63.4.42
        remote failure: 
        Successfully verified that the host, 10.63.4.42, is not the local machine as required.
        Successfully resolved host name to: /10.63.4.42
        Successfully connected to DCOM Port at port 135 on host 10.63.4.42.
        Successfully connected to NetBIOS Session Service at port 139 on host 10.63.4.42.
        Successfully connected to Windows Shares at port 445 on host 10.63.4.42.
        The remote file, C:\crap doesn't exist on 10.63.4.42 : The parameter is incorrect.
        
        Command validate-dcom failed.
        

        C:\crap is a directory present on the remote machine. I haven't set it up to be shared in any way, but I haven't done anything else to it, either. Any path supplied to --remotetestdir is considered to not exist. I've tried moving slashes around and doubling up backslashes in case it's a path issue; it's not.

        Hope this data point helps.

        Show
        ljnelson added a comment - - edited I have exactly the same problem. I installed and ran setup-local-dcom on the remote machine as an administrator. It claimed it ran successfully. Then I made sure that your steps 1-3 above were taken. I had to manually start the remote registry service. My remote machine is running Windows 7 Professional on a 64-bit machine with all updates installed. Here is my command and output: ljnelson$ asadmin --passwordfile ~/.glassfish.passwords --port=9048 validate-dcom --windowsuser lnelson --windowsdomain jenzabar --remotetestdir 'C:\crap' --verbose true 10.63.4.42 remote failure: Successfully verified that the host, 10.63.4.42, is not the local machine as required. Successfully resolved host name to: /10.63.4.42 Successfully connected to DCOM Port at port 135 on host 10.63.4.42. Successfully connected to NetBIOS Session Service at port 139 on host 10.63.4.42. Successfully connected to Windows Shares at port 445 on host 10.63.4.42. The remote file, C:\crap doesn't exist on 10.63.4.42 : The parameter is incorrect. Command validate-dcom failed. C:\crap is a directory present on the remote machine. I haven't set it up to be shared in any way, but I haven't done anything else to it, either. Any path supplied to --remotetestdir is considered to not exist. I've tried moving slashes around and doubling up backslashes in case it's a path issue; it's not. Hope this data point helps.
        Hide
        lb54 added a comment -

        Hi.
        I have also this issue:
        Win 2003 SP2 (Domain Admin Server, GF 3.1.1, updated to 3.1.2)
        Win 2008 Server R2 Enterprise SP1 (node, formerly connected through SSH via cygwin)
        User is authorized for both machines. DCOM is planned to replace the SSH-communication.

        Message is from Web Console is:
        Successfully verified that the host, myserver.host.xx, is not the local machine as required. Successfully resolved host name to: myserver.host.xx/<IP-Address> Successfully connected to DCOM Port at port 135 on host myserver.host.xx. Successfully connected to NetBIOS Session Service at port 139 on host gibson-10.tecis.hh. Successfully connected to Windows Shares at port 445 on host myserver.host.xx. The remote file, C: doesn't exist on myserver.host.xx : Logon failure: unknown user name or bad password.

        The CLI also fails with:
        remote failure: Command install-node-dcom failed.

        com.sun.enterprise.util.cluster.windows.process.WindowsException: Logon failure: unknown user name or bad password.
        Command create-node-dcom failed.

        Is there a way to "workaround" this or do I have to wait for an update?

        Show
        lb54 added a comment - Hi. I have also this issue: Win 2003 SP2 (Domain Admin Server, GF 3.1.1, updated to 3.1.2) Win 2008 Server R2 Enterprise SP1 (node, formerly connected through SSH via cygwin) User is authorized for both machines. DCOM is planned to replace the SSH-communication. Message is from Web Console is: Successfully verified that the host, myserver.host.xx, is not the local machine as required. Successfully resolved host name to: myserver.host.xx/<IP-Address> Successfully connected to DCOM Port at port 135 on host myserver.host.xx. Successfully connected to NetBIOS Session Service at port 139 on host gibson-10.tecis.hh. Successfully connected to Windows Shares at port 445 on host myserver.host.xx. The remote file, C: doesn't exist on myserver.host.xx : Logon failure: unknown user name or bad password. The CLI also fails with: remote failure: Command install-node-dcom failed. com.sun.enterprise.util.cluster.windows.process.WindowsException: Logon failure: unknown user name or bad password. Command create-node-dcom failed. Is there a way to "workaround" this or do I have to wait for an update?
        Hide
        jp2011 added a comment -

        There has been no fix for this because the cause is still unknown to Oracle. The workaround is do not use DCOM. We have personally abandoned Windows as a platform for production/QA in favour of RHEL 5 Linux distro. SSH is built in, and the cluster runs a lot faster with less overhead. The downside is that you have to learn Linux commands. But really, is this that bad?

        Show
        jp2011 added a comment - There has been no fix for this because the cause is still unknown to Oracle. The workaround is do not use DCOM. We have personally abandoned Windows as a platform for production/QA in favour of RHEL 5 Linux distro. SSH is built in, and the cluster runs a lot faster with less overhead. The downside is that you have to learn Linux commands. But really, is this that bad?
        Hide
        lb54 added a comment -

        I agree with you.
        BUT: Telling my company to use Linux Servers instead of Windows will not work, they don't want to hear that.
        Using SSH Nodes on Windows System with cygwin seems to be an alternative. But I used Glassfish 3.1.1 with ssh (cygwin) already, the communication seems to be not very stable (long running startup processes and long loading "Clusters" page with the Web Console).

        @Byron: Is there a plan for this bugfix so far?

        Show
        lb54 added a comment - I agree with you. BUT: Telling my company to use Linux Servers instead of Windows will not work, they don't want to hear that. Using SSH Nodes on Windows System with cygwin seems to be an alternative. But I used Glassfish 3.1.1 with ssh (cygwin) already, the communication seems to be not very stable (long running startup processes and long loading "Clusters" page with the Web Console). @Byron: Is there a plan for this bugfix so far?
        Hide
        mr_daemon added a comment -

        I did some incredibly tedious debugging and was able to get it to work:

        For the validate-dcom test to pass, since it seems to ignore the parameter for the test directory entirely and always use C:\ regardless, you must disable the new (vista+) policy that prevents users from elevating their privileges over the network by navigatinig to

        HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System

        and creating a new DWORD named LocalAccountTokenFilterPolicy of 1. This will allow the delete-me.bat file to be created there.

        However this then breaks again:

        PS D:\private> d:\glassfish3\bin\asadmin.bat --passwordfile dcom-pw.txt validate-dcom -w glassfish -v=true qlsvrnode2
        remote failure:
        Successfully verified that the host, qlsvrnode2, is not the local machine as required.
        Successfully resolved host name to: qlsvrnode2/192.168.9.11
        Successfully connected to DCOM Port at port 135 on host qlsvrnode2.
        Successfully connected to NetBIOS Session Service at port 139 on host qlsvrnode2.
        Successfully connected to Windows Shares at port 445 on host qlsvrnode2.
        Successfully accessed C: on qlsvrnode2 using DCOM.
        Successfully wrote delete_me.bat to C: on qlsvrnode2 using DCOM.
        Could not connect to WMI (Windows Management Interface) on qlsvrnode2. : Error setting up remote connection to WMI
        

        This is not mentionned at all in the documentation, but turns out you also need to change ownership and set permissions to the following registry key, in addition to the ones already listed:

        HKEY_CLASSES_ROOT\CLSID\{76A64158-CB41-11D1-8B02-00600806D9B6}
        

        Once this is accomplished, everything works as advertised.

        I am not fond of the security implications but at least it works and is at least more reliable than Cygwin+sshd.

        Show
        mr_daemon added a comment - I did some incredibly tedious debugging and was able to get it to work: For the validate-dcom test to pass, since it seems to ignore the parameter for the test directory entirely and always use C:\ regardless, you must disable the new (vista+) policy that prevents users from elevating their privileges over the network by navigatinig to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System and creating a new DWORD named LocalAccountTokenFilterPolicy of 1. This will allow the delete-me.bat file to be created there. However this then breaks again: PS D:\ private > d:\glassfish3\bin\asadmin.bat --passwordfile dcom-pw.txt validate-dcom -w glassfish -v= true qlsvrnode2 remote failure: Successfully verified that the host, qlsvrnode2, is not the local machine as required. Successfully resolved host name to: qlsvrnode2/192.168.9.11 Successfully connected to DCOM Port at port 135 on host qlsvrnode2. Successfully connected to NetBIOS Session Service at port 139 on host qlsvrnode2. Successfully connected to Windows Shares at port 445 on host qlsvrnode2. Successfully accessed C: on qlsvrnode2 using DCOM. Successfully wrote delete_me.bat to C: on qlsvrnode2 using DCOM. Could not connect to WMI (Windows Management Interface) on qlsvrnode2. : Error setting up remote connection to WMI This is not mentionned at all in the documentation, but turns out you also need to change ownership and set permissions to the following registry key, in addition to the ones already listed: HKEY_CLASSES_ROOT\CLSID\{76A64158-CB41-11D1-8B02-00600806D9B6} Once this is accomplished, everything works as advertised. I am not fond of the security implications but at least it works and is at least more reliable than Cygwin+sshd.
        Hide
        Byron Nevins added a comment -

        Thanks for the excellent comments and work everyone. I'll try and address this problem soon.

        Show
        Byron Nevins added a comment - Thanks for the excellent comments and work everyone. I'll try and address this problem soon.
        Hide
        lb54 added a comment -

        Hi Byron.
        Are there any plans to release this fix so far? Or is the "hack" described above the official solution?

        Thanks for info.

        Best wishes.

        Basti

        Show
        lb54 added a comment - Hi Byron. Are there any plans to release this fix so far? Or is the "hack" described above the official solution? Thanks for info. Best wishes. Basti
        Hide
        lb54 added a comment -

        Hi there.
        It seems that no one is working on this ticket right now.
        Is there a chance to get a fix for this in the near future?
        Unfortunatly the "quick fix" described above does not work for me, so I need another workaround or this bug fixed.

        Can anyone help me?

        Thanks.

        Basti

        Show
        lb54 added a comment - Hi there. It seems that no one is working on this ticket right now. Is there a chance to get a fix for this in the near future? Unfortunatly the "quick fix" described above does not work for me, so I need another workaround or this bug fixed. Can anyone help me? Thanks. Basti
        Hide
        mtobler added a comment - - edited

        I have not been able to get this to work on a set of 2008 R2 Servers
        which I am trying to cluster. Unfortunately I am unable to get the ssh
        functionality to work as well which leaves me with no clustering
        capability and wondering why we used Glassfish.
        Is anyone going to work on this anytime soon?

        I added the following to 18327 but am adding it here as requested:
        asadmin> validate-dcom --passwordfile do-not-delete gf01
        remote failure:
        Successfully verified that the host, gf01, is not the local machine as required.
        Successfully resolved host name to: gf01/172.18.11.169
        Successfully connected to DCOM Port at port 135 on host gf01.
        Successfully connected to NetBIOS Session Service at port 139 on host gf01.
        Successfully connected to Windows Shares at port 445 on host gf01.
        The remote file, C: doesn't exist on gf01 : Logon failure: unknown user name or bad password.

        I am using a domain and the user is a domain user.

        I have gone through every document I can find on this issue and have verified all settings/registry keys/etc are correct. I have tried this via asdamin and via the console and get the same result.

        Show
        mtobler added a comment - - edited I have not been able to get this to work on a set of 2008 R2 Servers which I am trying to cluster. Unfortunately I am unable to get the ssh functionality to work as well which leaves me with no clustering capability and wondering why we used Glassfish. Is anyone going to work on this anytime soon? I added the following to 18327 but am adding it here as requested: asadmin> validate-dcom --passwordfile do-not-delete gf01 remote failure: Successfully verified that the host, gf01, is not the local machine as required. Successfully resolved host name to: gf01/172.18.11.169 Successfully connected to DCOM Port at port 135 on host gf01. Successfully connected to NetBIOS Session Service at port 139 on host gf01. Successfully connected to Windows Shares at port 445 on host gf01. The remote file, C: doesn't exist on gf01 : Logon failure: unknown user name or bad password. I am using a domain and the user is a domain user. I have gone through every document I can find on this issue and have verified all settings/registry keys/etc are correct. I have tried this via asdamin and via the console and get the same result.
        Hide
        Byron Nevins added a comment -

        Sorry I overlooked the activity on this issue. I'll try to look into it soon. mtobler – please document what you did/what happened etc. Are you using a Windows Domain?

        Show
        Byron Nevins added a comment - Sorry I overlooked the activity on this issue. I'll try to look into it soon. mtobler – please document what you did/what happened etc. Are you using a Windows Domain?

          People

          • Assignee:
            Byron Nevins
            Reporter:
            jp2011
          • Votes:
            5 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated: