php - Quickest most efficient way to run a loop to find and verify over 1 million usernames off a different website? -
using php current script work it's going take extremely long time since there on 1 million usernames verify.
this checks against external websites api , returns <span class="user">{userid}</span>
if it's valid username , returns <span class="user">0</span>
if it's not.
$query = $db->query('select id, username users_to_verify'); // on 1 million. foreach($query $row) { $userid = $row['id']; $username = $row['username']; if(!preg_match("/<span class=\"user\">0<\/span>/", file_get_contents("http://website.net/api.php?username=".$username))) $db->query('update users_to_verify set verified = 1 id = $userid'); } }
is quickest way this? i've dabbled curl both file_get_contents , curl seem have same performance, know of dependent on external websites response time, want make sure side using quickest , best approach possible.
i don't know file_get_contentes
, curl
performance. can boost speed run query in 1 command :
$query = $db->query('select id, username users_to_verify'); // on 1 million. $update_query = []; foreach($query $row) { $userid = $row['id']; $username = $row['username']; if(!preg_match("/<span class=\"user\">0<\/span>/", file_get_contents("http://website.net/api.php?username=".$username))) $update_query[] = 'update users_to_verify set verified = 1 id = $userid'; } } $db->query(implode(';',$update_query));
in case don't have connect , run query in every step of loop.
edit:
it better collect verified userids , use where id in(implode(',', $userids))
. mysql has handle single query.
$query = $db->query('select id, username users_to_verify'); // on 1 million. $userids = []; foreach($query $row) { $userid = $row['id']; $username = $row['username']; if(!preg_match("/<span class=\"user\">0<\/span>/", file_get_contents("http://website.net/api.php?username=".$username))) $userids[] = $userid; } } $db->query('update users_to_verify set verified = 1 id in('.implode(',', $userids).')');
Comments
Post a Comment