Arduino Ethernet wierdness

[BrianDagobah]

I'm stumped. I'm hoping someone here as some insight into why I'm experiencing what I'm experiencing.

The heck am I trying to do?

Part one:
I have an array of sensors that I'm reading. Temp, light, and various other analog sensors. I'm taking those values and serving up a webpage to display them. You can see it live here: http://garden.dagobah-system.com/  tadaa! it's works see. Happy webpage. Refresh manually and you can see the data update. For the record, that is being served up directly from my Arduino. That webpage lives on the arduino. I'm not copying txt files to an Apache or IIS server or anything. That's all Arduino right there.

Part two:
Part two of this little gizmo, goes out to a "real" webserver (IIS, server in my house, on the network, etc..) and looks at a page that looks like this.



Yup. It's a webpage that displays the number zero. It's literally a txt file with 1 character in it, saved as a .html file. This file lives on a local IIS server.

OH, sometimes it looks like this:

The number 1. Same file, just the content changes. It changes from a 1 to a 0. Right now I'm changing it manually for testing. eventually it's state will be changed by a.....blah blah blah. I'll share when I'm done and it works.

The code on the arduino has 4 functions.
1. read the sensors: - Check! Works great.
2. Serve up that webpage: - Check! works great.
3. READ that 1 or 0 from that other page. -Check! Works great
4. Turn on an LED (place holder for a 12V water valve) if function 3 =1, turn it off if function 3 = 0. Check. Works great!

I can let this run, and change that 1 to a 0 for days and it just continues to work. LED turns on and off, the sensors are all read properly and verified through the webpage and the serial monitor debug function I added.

The problem:
If I view that first webpage (the one with all the temps on it-served up from the Arduino directly) from a PC inside my network, the Arduino freaks the HECK out. For some reason, it locks up and will no longer READ the 1 or 0 webpage. It continues to read and display the temps, but it stops properly reading the 1 or 0 site served up on my local IIS server. If I reset the ethernet shield connected to the arduino, it starts working again....untill I view and refresh the data page served up by the arduino directly.

The kicker, is if I view the page from outside my network (RDP'd to a server at my office or on my cellphone for example) the Arduino is fine. It's ONLY if I view/refresh the data page from inside my network. Crazy.

Anyway, I've been chasing this problem for months. I'm happy to post code, in follow-up posts once I've properly commented it, but this seems like something characteristic of the Ethernet Shield. I want SOOO badly to blame this on IIS, but I don't think it's the issue. I think it's something related to the Ethernet shield getting a request while it's trying to READ a page... but why only on the local network is this an issue?  Anyway, I'm completely stumped.

Anyone have any ideas? I REALLY don't want to use 2 arduinos with 2 Ethernet shields.

Thanks,
Brian

[BrianDagobah]

The code...with it mostly commented. I'm NOT...NOT much of a coder, so please be gentle.

Code: [Select]

#include <SPI.h>
#include <Ethernet.h>
//so, most of this first part is direct from the Arduino webserver example.
byte mac[] = {
  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress ip(10,10,27,20);
IPAddress dnserver(10,10,27,5);
IPAddress gateway(10,10,27,1);
char serverName[] = "master-yoda.dagobahsystem.local";
EthernetClient client;

//global Variables I setup.
int analogChannel0 = 0;
int analogChannel1 = 1;
int analogChannel2 = 2;
char valve_status_1;
float tempF0;
float tempF1;
float light1;
EthernetServer server(80);

void setup() {
 // Open serial communications and wait for port to open:
  Serial.begin(9600);
   

//more stuff from the Arduino Example.

  // start the Ethernet connection and the server:
  Ethernet.begin(mac, ip, dnserver, gateway);
  server.begin();
  Serial.print("Garden IP Address is: ");
  Serial.println(Ethernet.localIP());
}

void loop(){
  //the core functions of this whole sketch.
  ReadSensors(); //reads all the sensors attached to the Arduino
  ServPage();  //displays the values from the first function on a webpage
  DebugSerial(); //this just sends a whole bunch of nerdy stuff to the serial monitor so I can see what the heck is going on.
  ReadCtrlPage();  // this goes out and reads the 1 or 0 from my IIS server page. The function does stuff, but that stuff will be moved to the dostuff() function - still troubleshooting
  //DoStuff();   //this WILL be the function that carrys out all the tasks that happen as a result of the information from the first 4 functions... and some external sources as well.
}

void ReadSensors(){
//This function reads all the sensors, does some math and stores the results in global variables.
 
  //Read Temp Sensor 0 on pin A0

  long sum = 0;         //  *****
  float voltage;        //  Local
  float temperature;    //  Variables
  int lsbCount;         //  *****
  float lsb = 1.23432/1023; // This is a varable associated with a precision voltage regulator that I don't have connected yet.
 
  for (int i = 1; i <= 10; i++)
  sum += analogRead(analogChannel0);    // sum 10 readings
  lsbCount = (sum + 5) / 10;                  // and average
  voltage = lsbCount * lsb;
  temperature = (voltage / 0.01) - 10.0;  //temp in C
  tempF0 = ((temperature * 1.8) + 32);    // Converts to F
 
 
  //Read Temp Sensor 1 on pin A1
  sum = 0;
  for (int i = 1; i <= 10; i++)
  sum += analogRead(analogChannel1);    // sum 10 readings
  lsbCount = (sum + 5) / 10;                  // and average
  voltage = lsbCount * lsb;
  temperature = (voltage / 0.01) - 10.0; //same as the previous function
  tempF1 = ((temperature * 1.8) + 32);  //same as the previous function
 

  sum = 0;
  for (int i = 1; i <= 10; i++)
  sum += analogRead(analogChannel2);    // sum 10 readings
  lsbCount = (sum/10);                  // and average
  voltage = lsbCount * 5.0;
  voltage /= 1024.0;
  light1 = voltage;      //I have no idea how to conver this to Lux or candle power or lumins or whatever. so I'm leaving it volts for now.

}




void ServPage(){
//this function generates the webpage at http://garden.dagobah-system.com
 
//The next 20 lines are pretty much copy/paste from the Arduino IDE webserver example. 
  EthernetClient client = server.available();
  if (client) {
    Serial.println("new client");
    // an http request ends with a blank line
    boolean currentLineIsBlank = true;
    while (client.connected()) {
      if (client.available()) {
        char c = client.read();
        Serial.write(c);
        // if you've gotten to the end of the line (received a newline
        // character) and the line is blank, the http request has ended,
        // so you can send a reply
        if (c == '\n' && currentLineIsBlank) {
          // send a standard http response header
          client.println("HTTP/1.1 200 OK");
          client.println("Content-Type: text/html");
          client.println("Connnection: close");
          client.println();
          client.println("<!DOCTYPE HTML>");
          client.println("<html>");
          //I took out the meta refresh because it kept breaking my Arduino... so now I refresh manually for testing.. :(
                    // add a meta refresh tag, so the browser pulls again every 5 seconds:
          //client.println("<meta http-equiv=\"refresh\" content=\"5\">");
       
          //*********************************************************************
          //*                  LETS DISPLAY SOME STUFF!!!!!                     *
          //*********************************************************************
         
         //the next like million lines of code is me doing HTML codeing via client.print(). I know I can consolidate this a little bit but the way I have it here makes it easier to read/edit I think.
         
         client.print("<table bgcolor=#55FFAA border=1>");
         client.print("<tr>");
         client.print("<td width=267><b><center>INPUT SENSOR DATA</center></b></td>");
         client.print("</tr>");
         client.print("</table>");
         //Table Setup Format
         // Sensor Name, Value, Unit, ADC Channel
         client.print("<table border=1 >");
         client.print("<tr>");
         client.print("<td>");
         client.print("<b>Sensor Name</b>");
         client.print("</td>");
         client.print("<td>");
         client.print("<b>Value</b>");
         client.print("</td>");
         client.print("<td>");
         client.print("<b>Unit</b>");
         client.print("</td>");
         client.print("<td>");
         client.print("<b>ADC Channel</b>");
         client.print("</td>");
         client.print("</tr>");
         
         //DATA
         client.print("<tr>");
         client.print("<td>");
         client.print("Shady Temp ");
         client.print("</td>");
         client.print("<td>");
         client.print(tempF0);
         client.print("</td>");
         client.print("<td>");
         client.print(" F");
         client.print("</td>");
         client.print("<td>");
         client.print(analogChannel0);
         client.print("</td>");
         client.print("</tr>");
         
         client.print("<tr>");
         client.print("<td>");
         client.print("Sunny Temp ");
         client.print("</td>");
         client.print("<td>");
         client.print(tempF1);
         client.print("<td>");
         client.print(" F");
         client.print("</td>");
         client.print("<td>");
         client.print(analogChannel1);
         client.print("</td>");
         client.print("</tr>");
         
         client.print("<tr>");
         client.print("<td>");
         client.print("Ambiant Light ");
         client.print("</td>");
         client.print("<td>");
         client.print(light1);
         client.print("</td>");
         client.print("<td>");
         client.print(" V");
         client.print("</td>");
         client.print("<td>");
         client.print(analogChannel2);
         client.print("</td>");
         client.println("<tr/>");   
         client.println("</table>");
         
         client.print("<table bgcolor=#55FFAA border=1>");
         client.print("<tr>");
         client.print("<td width=267><b><center>OUTPUT CONTROL STATUS</center></b></td>");
         client.print("</tr>");
         client.print("</table>");
         
         
         client.print("<table border=1 >");
         client.print("<tr>");
         client.print("<td>");
         client.print("<b>Output Name</b>");
         client.print("</td>");
         client.print("<td>");
         client.print("<b>Value</b>");
         client.print("</td>");
         client.print("<td>");
         client.print("<b>Unit</b>");
         client.print("</td>");
         client.print("<td>");
         client.print("<b>I/O Channel</b>");
         client.print("</td>");
         client.print("</tr>");
         
         
         client.print("<tr>");
         client.print("<td>");
         client.print("Valve 1");
         client.print("</td>");
         client.print("<td>");
         client.print(valve_status_1);
         client.print("</td>");
         client.print("<td>");
         client.print("1/0");
         client.print("</td>");
         client.print("<td>");
         client.print(analogChannel2);
         client.print("</td>");
         client.println("<tr/>"); 
         client.print("</table>");
         
         
         

          //*********************************************************************
          //*                  ...AND WE'RE DONE DISPLAYING                     *
          //*********************************************************************

          //The rest of this function is right out of the Arduino Example too. I kinda know what's going on, but... it's best if I leave it alone.
          client.println("</html>");
                           
         
          break;
        }
        if (c == '\n') {
          // you're starting a new line
          currentLineIsBlank = true;
        }
        else if (c != '\r') {
          // you've gotten a character on the current line
          currentLineIsBlank = false;
        }
      }
    }
    // give the web browser time to receive the data
    delay(1);
    // close the connection:
    client.stop();
    Serial.println("client disonnected from browser request");
    delay(1000);
      }
    }


void ReadCtrlPage(){
//This function goes and reads that IIS page looking for a 1 or 0.
  client.stop();  //I put this in here for troubleshooting to make sure that the ethernet shield is not doing anything until I tell it to.
 delay(1000); //again, more trouble shooting. figured I'd give it second, to close any open connections. I dunno. Seemed reasonable.
 Serial.println("connecting to control page..."); //just for the serial monitor. Let'n yall know what's going on.
 
    if (client.connect(serverName, 80)) { //connects to the IIS server on port 80
    Serial.println("connected"); 

// Make a HTTP request:
    client.println("GET http://master-yoda.dagobahsystem.local/do.html"); //sends the request via the client function to the IIS web server looking for a page called do.html
    Serial.println("GET http://master-yoda.dagobahsystem.local/do.html"); //displays the same thing to the serial monitor.
    client.println("HTTP/1.1");
    Serial.println("HTTP/1.1");
    client.println("Host: master-yoda.dagobahsystem.local");
    Serial.println("Host: master-yoda.dagobahsystem.local");
    client.println();
    Serial.println();
    }
  else {
    // if you didn't get a connection to the server it just says this, then moves on.
    Serial.println("connection failed");
    Serial.println("We'll give it another go on the next go-around.");
    delay(2000); //so you can read it.
      } 

if (client.available()) {
    char d = client.read();  //reads the page, and stores the character it finds in the variable "d".
    Serial.println(d); //prints "d" to the serial monitor so I can verify what it found out there.
    Serial.println();
          if (d == '1'){  //this will be move to the "dostuff()" function eventually.
      digitalWrite(7,HIGH); //if d = 1 then turn on the LED on digital pin 7.
    }
    else if(d = '0'){  //if d = 0 turn the LED off.
      digitalWrite(7,LOW);
    }
    valve_status_1 = d; //takes the local variable "d" and stores it in the global variable "valve_status_1" so I can use it other places.
   client.stop(); //stops the client function.
  }

  // if the server's disconnected, stop the client:
  if (!client.connected()) {
    Serial.println();
   Serial.println("disconnecting.");
   client.stop();
  }

}


void DebugSerial(){
//this is my debug function. I'll comment it out once everything is working, but for now I like having it.

  Serial.println("************* ");
  Serial.print("temp 0: ");
  Serial.println(tempF0);
  Serial.print("temp 1: ");
  Serial.println(tempF1); 
  Serial.print("light 1: ");
  Serial.println(light1);
  Serial.println(valve_status_1);
  delay(3000);
}

[Psi]

This is just a stab in the dark. but..

It could be your web browser attempts to establish a HTTPS connection first before falling back to HTTP and the arduino doesn't the HTTPS request.

When connecting from outside of your LAN the https port will be blocked by NAT so doesn't get to the arduino.

Maybe try typing the full HTTP:// address into the browser and see if you get the same problem.
(im assuming you've just been typing the DNS name or IP with no HTTP ?)

[BrianDagobah]

OH! I hope it's something simple like that... I'll try that and see if gets around the issue.

Brian

[BrianDagobah]

Nope. It broke. So weird. I can refresh that page from my office 100 times if I want. Hit it once from inside and BZZZZZZZZ... broke.

Brian

[mrflibble]

Maybe the difference is that having the 2nd PC in the local network is causing extra windows domain bullshit broadcasts, that are not being sent in the single server case. If your arduino barfs on the broadcasts then that would explain it.

Well that, or having the 2nd PC and the arduino on the same IP address or even mac address ... but that would be really silly.

[Psi]

Do you have any old win98 computers you could test out on the local LAN and see if they have the problem?

It might help prove something.

You could also try using a smartphone over wifi.

[IanJ]

I seem to remember something similar................is this helpful?

http://forum.arduino.cc/index.php/topic,114291.0.html

Ian.

[BrianDagobah]

Maybe the difference is that having the 2nd PC in the local network is causing extra windows domain bullshit broadcasts, that are not being sent in the single server case. If your arduino barfs on the broadcasts then that would explain it.

Well that, or having the 2nd PC and the arduino on the same IP address or even mac address ... but that would be really silly.

I suppose it's possible. I assure you I don't have duplicate MACs or IPs, but the broadcast thing is curious enough. When I get home tonight, I'll put that Arduino on a separate VLAN. That layer 3 component should block any broadcasts. Probably not a bad idea to isolate my projects like that anyway. Some nerd will figure out how to hack my network through my Arduino.

THanks,
Brian

[BrianDagobah]

Do you have any old win98 computers you could test out on the local LAN and see if they have the problem?

It might help prove something.

You could also try using a smartphone over wifi.

I could spin up a Win98 VM I suppose. I'll verify the smartphone thing, but I'm pretty sure that broke it too. I also have a Chromebook and an iPad I could try as well. Heck, maybe it is an OS thing. I'm willing to try anything at this point.

Thanks,
Brian

[BrianDagobah]

I seem to remember something similar................is this helpful?

http://forum.arduino.cc/index.php/topic,114291.0.html

Ian.

That's interesting, but I'm not sure we're having the same issue. Although! I'm not sure why I never thought to look at wireshark..https://www.eevblog.com/forum/Smileys/default/facepalm.gif DUH. This could REALLY give me some insight.

Ok, well, when I get home tonight, I'm going to check out the traffic with WireShark, if that doesn't make the solution obvious, then I'll try the VLAN thing, if THAT doesn't work. I will watch TV and drink a beer.

Brian

[amyk]

I'm also a little surprised why no one else mentioned getting out the wireshark... as someone who has debugged a lot of embedded networked systems, it is the #1 tool to use.

And judging by your symptoms, I would look in more detail into ARP.

[BrianDagobah]

I also use WireShark everyday. No idea why it didn't occur to me  to use it on this... LOL. I guess it's the work/hobby disconnect. eh? I'm still looking for some way to generate SNMP data with my arduino. That's another project though.

Brian

[BrianDagobah]

OK. Sorry it took so long to reply to this. Stupid job/life getting in the way.

So, WireShark yielded nothing useful that I could tell (dug out an old ethernet hub and connected it to the Arduino and my Wireshark laptop). However, creating another VLAN seemed to work. As long as there is a L3 "barrier" between the Arduino and the PC hitting that website, everything works fine. So, my hobby projects now have their own VLAN and associated ACL. How about that! It's GOT to be broadcasts or something. Not sure where the problem is, but meh... chalk it up to, weird crap happens I suppose. When I migrate this all over to my Linux VM I'll try putting it back on the original VLAN and see what happens. Maybe it is IIS...

Next step: get SNMP data out of the Arduino ethernet shield.

Thanks
 

[westfw]

WireShark yielded nothing useful that I could tell

But what does it show?  Does the Arduino respond to the ARP request from the local system?  Does it (in turn) ARP FOR the address of the local system?  Does it send IP packets for the local system to the proper ethernet address?  Which side sends (or fails to send) the first "mistake"?  Can you duplicate the bad behavior if you make a telnet connection to the web server address, or is it only a problem with a browser?

In theory, I should be doing more with Ethernet Arduinos.  In reality, I'm terrified of using a TCP/IP implementation that has no "internals visibility."  (shudder.)