c# - Replicating JSP/JSF/IceFaces requests for scraping purposes -

-

i trying scrape data off enterprise website made using jsf , icefaces. using c# , restsharp library.

i have no experience jsp , jsf or icefaces @ all, trying figure out how replicate site doing using http requests wasn't successful. site not have concept of routing whatsoever (and when accidentally happen press button in browser, logged out...).

what have managed far:

  • make post request credentials /login resource in order log in
  • retrieve jsessionid cookie after login , store cookiecontainer
  • use regexes ice.session , ice.view values
  • replicate post request block/send-receive-updates

if original post request managed js code on site (when clicking around), returns xml response this: 

<updates>     <update address="some form id" tag="table"> ... </update>     <update address="content" tag="div"> ... </update>     <update address="the ice.session id, ice.view number separated : followed string 'dynamic-code'" tag="script"> ... </update> </updates>

however, took encoded post params request doing on site , replicated them in c# code , response has last update (script tag) this:

<updates>     <update address="the ice.session id, ice.view number separated : followed string 'dynamic-code'" tag="script"> ... </update> </updates>

does please have experience scraping/testing these technologies , can me figure out doing wrong ?

thanks.


Comments

Post a Comment

Popular posts from this blog

Using django-mptt to get only the categories that have items -

-
sample structure: all uppercase: category mixed case: item root ├── books │   ├── fiction │   │   └── classics │   └── non-fiction ├── clothing └── electronics ├── laptops ├── phones │   ├── apple │   │   ├── iphone 6 │   │   ├── iphone 6 plus │   │   ├── iphone 6s │   │   └── iphone 6s plus │   ├── motorola │   │   ├── moto g4 │   │   ├── moto g4 play │   │   ├── moto g4 plus │   │   └── moto x │   └── samsung └── tablets └── apple i'm trying have index page show "electronics" category. "clothing" category shouldn't show since it's empty , "books" category shouldn't show either since, although has child categories, child categories don't have items. similarly, "electronics" page should show "phones" category. "laptops" category shouldn't show since it's empty , "tablets" category shouldn't show either since...
Read more

ruby - Train neural network with sine function -

-
i want train neural network sine() function. currently use code , ( cerebrum gem): require 'cerebrum' input = array.new 300.times |i| inputh = hash.new inputh[:input]=[i] sinus = math::sin(i) inputh[:output] = [sinus] input.push(inputh) end network = cerebrum.new network.train(input, { error_threshold: 0.00005, iterations: 40000, log: true, log_period: 1000, learning_rate: 0.3 }) res = array.new 300.times |i| result = network.run([i]) res.push(result[0]) end puts "#{res}" but not work, if run trained network weird output values (instead of getting part of sine curve). so, doing wrong? cerebrum basic , slow nn implementation. there better options in ruby, such ruby-fann gem. most problem network simple. have not specified hidden layers - looks code assigns default hidden layer 3 neurons in case. try like: network = cerebrum.new({ learning_rate: 0.01, momentum: 0.9, hidden_l...
Read more

javascript - Java with RWeka packege -

-
i have mac el capitan, 10.11.5, used rweka package, downloaded correctly , initialized error, when apply j48(species ~ ., data = iris) i have error error in .jnew("weka/core/attribute", attname[i], .jcast(levels, "java/util/list")) : java.lang.unsupportedclassversionerror: weka/core/attribute : unsupported major.minor version 51.0 i used java version these details, 52f85:~ kameljabreen$ java -version java version "1.7.0_79" java(tm) se runtime environment (build 1.7.0_79-b15) java hotspot(tm) 64-bit server vm (build 24.79-b02, mixed mode) please me solve problems, problm since 1 month, read lot problem in web sites. did thing, installing java jdk version 1.8 .... ect. used java 1.7. please give me solutions. in advance the rweka package depends on rjava package r-to-java communication, includes , uses own java 1.6 runtime. make things worse, behavior of rjava package platform dependent - uses "built-in" java on p...
Read more