<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
	<title type="html"><![CDATA[Серый форум &mdash; JScript:Быстрый поиск узла в большом XML файле]]></title>
	<link rel="self" href="https://forum.script-coding.com/extern.php?action=feed&amp;tid=9303&amp;type=atom" />
	<updated>2014-03-04T09:11:50Z</updated>
	<generator>PunBB</generator>
	<id>https://forum.script-coding.com/viewtopic.php?id=9303</id>
		<entry>
			<title type="html"><![CDATA[Re: JScript:Быстрый поиск узла в большом XML файле]]></title>
			<link rel="alternate" href="https://forum.script-coding.com/viewtopic.php?pid=80513#p80513" />
			<content type="html"><![CDATA[<p>Не ожидано нашел дополнительное ускорение поиска узлов в обновляемом XML файле.<br />Теперь реальный скрипт (написанный в 2006г) стал&nbsp; работать в 30 раз быстрее (с 110 минут до 3.5 минут)<br />Примерный Алгоритм:<br />1.Читаем XML файл<br />2. Заполняем <br />&nbsp; &nbsp; &nbsp;Объект&nbsp; значениями ключей и порядковым номером узла o_i[хэш]=i<br />&nbsp; &nbsp; &nbsp;Создаем массив куда копируем удаленные из XML узлы </p><div class="codebox"><pre><code>a_i[i]=  oNodes[i].parentNode.removeChild(oNodes[i])</code></pre></div><p>3. Выполняем массовую обработку<br />4. Востанавливаем XML файл<br /></p><div class="codebox"><pre><code>  for (var j = 0; j &lt; a_i.length; j++) {
    r.appendChild(a_i[j])
  }</code></pre></div><p>5. Сохраняем файл</p><div class="fancy_spoiler_switcher"><div class="fancy_spoiler_switcher_header"><strong>+</strong>&nbsp;Тест2</div><div class="fancy_spoiler"><div class="codebox"><pre><code>var fso = new ActiveXObject(&quot;Scripting.FileSystemObject&quot;)
var logFile = fso.OpenTextFile(&quot;test_2.log&quot;, 8, true)

var aStep = [10, 15, 25, 50, 150, 250, 100, 100]
var MaxNode = 0
for (var z = 0; z &lt; aStep.length; z++) {
  MaxNode += (aStep[z] * 1000)
  alert(&quot;узлов:&quot; + MaxNode)
  tt = new Date()
  t = new Date()
  dom = loadDom(&quot;test.xml&quot;)

  alert(&quot;1. Read XML         :&quot; + time(t))

  r = dom.documentElement
  t = new Date()
  var o_i = {}
  var a_i = []
  var sPath = &quot;//e&quot;

  t = new Date()
  var oNodes = dom.selectNodes(sPath);

  for (var i = 0; i &lt; oNodes.length; i++) {
    var e = oNodes[i]
    o_i[[e.getAttribute(&quot;i&quot;), e.getAttribute(&quot;k&quot;), e.getAttribute(&quot;l&quot;)].join(&quot;|&quot;)] = i
    a_i[i] = oNodes[i].parentNode.removeChild(oNodes[i]);

  }
  alert(&quot;2. Create index Node:&quot; + time(t))

  for (var j = 0; j &lt; MaxNode * 4; j++) {
    i = Math.round(Math.random() * MaxNode)
    e = null
    var e_num = -1
    var x = [i, i % 3, i % 7].join(&quot;|&quot;)
    if (o_i[x] == null) {
      o_i[x] = a_i.length
      e = dom.createElement(&quot;e&quot;)
      e.setAttribute(&quot;i&quot;, i)
      e.setAttribute(&quot;k&quot;, i % 3)
      e.setAttribute(&quot;l&quot;, i % 7)
      e.setAttribute(&quot;s&quot;, &quot;0&quot;)
      a_i[a_i.length] = e
    }
    e_num = o_i[x]
    e = a_i[o_i[x]]

    e.setAttribute(&quot;s&quot;, (1 * e.getAttribute(&quot;s&quot;)) + 1)
    f = e.appendChild(dom.createElement(&quot;f&quot;))
    f.setAttribute(&quot;ii&quot;, i)
    f.setAttribute(&quot;t&quot;, j)

    o_i[[e.getAttribute(&quot;i&quot;), e.getAttribute(&quot;k&quot;), e.getAttribute(&quot;l&quot;)].join(&quot;|&quot;)] = e_num &lt; 0 ? r.childNodes.length - 1 : e_num
  }
  alert(&quot;3. Append Node      :&quot; + time(t))
  t = new Date()

  for (var j = 0; j &lt; a_i.length; j++) {
    r.appendChild(a_i[j])
  }

  alert(&quot;4. Create XML       :&quot; + time(t) + lNode(dom))

  dom.save(&quot;test.xml&quot;)
  alert(&quot;5. Save XML         :&quot; + time(t))
  alert(&quot;               total:&quot; + time(tt))
  alert(&quot;&quot;)

  fso.CopyFile(&quot;test.xml&quot;, &quot;t.&quot; + MaxNode + &quot;.xml&quot;)

}

function alert(x) {
  WScript.Echo(x)
  logFile.WriteLine(x)
}

function loadDom(xFile) {
  var dom
  dom = new ActiveXObject(&quot;msxml2.DOMDocument.6.0&quot;);
  dom.async = false;
  dom.validateOnParse = false;
  dom.resolveExternals = false;
  if (!dom.load(xFile)) {
    dom.appendChild(dom.createProcessingInstruction(&quot;xml&quot;, &quot;version=&#039;1.0&#039; encoding=&#039;windows-1251&#039;&quot;))
    dom.appendChild(dom.createElement(&quot;root&quot;))
  }
  dom.setProperty(&quot;SelectionLanguage&quot;, &quot;XPath&quot;)
  return dom
}

function time(t) {
  var tt = new Date()
  var d = (new Date(tt - t))
  d.setMinutes(d.getMinutes() + d.getTimezoneOffset())
  return (&quot;&quot; + r(d.getHours()) + &quot;:&quot; + r(d.getMinutes()) + &quot;:&quot; + r(d.getSeconds()) + &quot;.&quot; + d.getMilliseconds())

    function r(d) {
      var s = &quot;00&quot; + d
      return (s.substr(s.length - 2, 2))
    }
}

function lNode(dom) {
  var sPath = &quot;//e&quot;
  var oNodes = dom.selectNodes(sPath);
  return (&quot; узлов:&quot; + oNodes.length)
}</code></pre></div></div></div><div class="fancy_spoiler_switcher"><div class="fancy_spoiler_switcher_header"><strong>+</strong>&nbsp;Результаты теста2</div><div class="fancy_spoiler"><div class="quotebox"><blockquote><p>узлов:10000<br />1. Read XML&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:00:00.0<br />2. Create index Node:00:00:00.0<br />3. Append Node&nbsp; &nbsp; &nbsp; :00:00:00.985<br />4. Create XML&nbsp; &nbsp; &nbsp; &nbsp;:00:00:00.15 узлов:9815<br />5. Save XML&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:00:00.125<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;total:00:00:01.110</p><p>узлов:25000<br />1. Read XML&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:00:00.93<br />2. Create index Node:00:00:00.157<br />3. Append Node&nbsp; &nbsp; &nbsp; :00:00:02.719<br />4. Create XML&nbsp; &nbsp; &nbsp; &nbsp;:00:00:00.63 узлов:24743<br />5. Save XML&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:00:00.485<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;total:00:00:03.297</p><p>узлов:50000<br />1. Read XML&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:00:00.359<br />2. Create index Node:00:00:00.406<br />3. Append Node&nbsp; &nbsp; &nbsp; :00:00:05.938<br />4. Create XML&nbsp; &nbsp; &nbsp; &nbsp;:00:00:00.125 узлов:49534<br />5. Save XML&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:00:01.62<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;total:00:00:07.359</p><p>узлов:100000<br />1. Read XML&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:00:01.281<br />2. Create index Node:00:00:00.844<br />3. Append Node&nbsp; &nbsp; &nbsp; :00:00:13.641<br />4. Create XML&nbsp; &nbsp; &nbsp; &nbsp;:00:00:00.266 узлов:99077<br />5. Save XML&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:00:01.891<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;total:00:00:16.813</p><p>узлов:250000<br />1. Read XML&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:00:05.47<br />2. Create index Node:00:00:01.765<br />3. Append Node&nbsp; &nbsp; &nbsp; :00:00:39.813<br />4. Create XML&nbsp; &nbsp; &nbsp; &nbsp;:00:00:00.641 узлов:247235<br />5. Save XML&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:00:03.531<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;total:00:00:48.391</p><p>узлов:500000<br />1. Read XML&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:00:27.32<br />2. Create index Node:00:00:04.672<br />3. Append Node&nbsp; &nbsp; &nbsp; :00:01:43.268<br />4. Create XML&nbsp; &nbsp; &nbsp; &nbsp;:00:00:01.344 узлов:495284<br />5. Save XML&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:00:05.656<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;total:00:02:15.971</p><p>узлов:600000<br />1. Read XML&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:01:56.456<br />2. Create index Node:00:00:09.765<br />3. Append Node&nbsp; &nbsp; &nbsp; :00:02:31.862<br />4. Create XML&nbsp; &nbsp; &nbsp; &nbsp;:00:00:01.750 узлов:598139<br />5. Save XML&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:00:08.672<br />&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;total:00:04:36.990</p><p>узлов:700000<br />1. Read XML&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:01:09.408<br />2. Create index Node:00:00:00.0&nbsp; -- Скрипт не смог считать узлы dom.selectNodes(sPath) вернул 0 Узлов</p></blockquote></div></div></div>]]></content>
			<author>
				<name><![CDATA[badik]]></name>
				<uri>https://forum.script-coding.com/profile.php?id=21834</uri>
			</author>
			<updated>2014-03-04T09:11:50Z</updated>
			<id>https://forum.script-coding.com/viewtopic.php?pid=80513#p80513</id>
		</entry>
		<entry>
			<title type="html"><![CDATA[JScript:Быстрый поиск узла в большом XML файле]]></title>
			<link rel="alternate" href="https://forum.script-coding.com/viewtopic.php?pid=80407#p80407" />
			<content type="html"><![CDATA[<p>Для нормального размера XML файлов метод selectSingleNode является незаменимым.<br />Однако если число узлов больше 200 000 узлов&nbsp; (50 000 первого&nbsp; и 150 000 второго порядков) поиск при обновлении XML начинает тормозить.<br />Если вместо selectSingleNode использовать Object (hash массив) для хранения ключей поиска и порядковых номеров узлов скорость поиска возрастет в разы до определеного предела.</p><div class="codebox"><pre><code>    dom = loadDom()
    r = dom.documentElement
    t = new Date()
    var o_i = {}
    var sPath = &quot;//e&quot;
    var oNodes = dom.selectNodes(sPath);
    for (var i = 0; i &lt; oNodes.length; i++) {
        o_i[[e.getAttribute(&quot;i&quot;), e.getAttribute(&quot;k&quot;), e.getAttribute(&quot;l&quot;)].join(&quot;|&quot;)] = i
    }

    for (var j = 0; j &lt; MaxNode * 4; j++) {
        i = Math.round(Math.random() * MaxNode)
        e = null
        var e_num = -1
        var x = [i, i % 3, i % 7].join(&quot;|&quot;)
        if (o_i[x] == null) {
            e = r.appendChild(dom.createElement(&quot;e&quot;))
            e.setAttribute(&quot;i&quot;, i)
            e.setAttribute(&quot;k&quot;, i % 3)
            e.setAttribute(&quot;l&quot;, i % 7)
            e.setAttribute(&quot;s&quot;, &quot;0&quot;)
        } else {
            e_num = o_i[x]
            e = r.childNodes[o_i[x]]
        }
        e.setAttribute(&quot;s&quot;, (1 * e.getAttribute(&quot;s&quot;)) + 1)
        f = e.appendChild(dom.createElement(&quot;f&quot;))
        f.setAttribute(&quot;ii&quot;, i)
        f.setAttribute(&quot;t&quot;, j)

        o_i[[e.getAttribute(&quot;i&quot;), e.getAttribute(&quot;k&quot;), e.getAttribute(&quot;l&quot;)].join(&quot;|&quot;)] = e_num &lt; 0 ? r.childNodes.length - 1 : e_num
    }</code></pre></div><p>В тесте используются<br /></p><ul><li><p>1.ADD - простое последовательное добавление узлов.</p></li><li><p>2.selectSingle - Случайное добавление узлов поиск selectSingleNode.</p></li><li><p>3.selectObject Случайноe добавление через Object hash массив.</p></li><li><p>4.Скорость заполнения hash массив XML полученный в 3 тесте.</p></li></ul><br /><div class="fancy_spoiler_switcher"><div class="fancy_spoiler_switcher_header"><strong>+</strong>&nbsp;Тест</div><div class="fancy_spoiler"><div class="codebox"><pre><code>var aCount = [1000, 5000, 10000, 25000, 40000, 50000, 75000, 100000, 150000, 200000]
for (var z = 0; z &lt; aCount.length; z++) {
    var MaxNode = aCount[z]
    WScript.Echo(&quot;узлов:&quot; + MaxNode)

    dom = loadDom()
    r = dom.documentElement
    t = new Date()
    for (var i = 0; i &lt; MaxNode; i++) {
        e = r.appendChild(dom.createElement(&quot;e&quot;))
        e.setAttribute(&quot;i&quot;, i)
        e.setAttribute(&quot;k&quot;, i % 3)
        e.setAttribute(&quot;l&quot;, i % 7)
        e.setAttribute(&quot;s&quot;, &quot;0&quot;)

        for (var j = 0; j &lt; 4; j++) {
            e.setAttribute(&quot;s&quot;, (x = 1 * e.getAttribute(&quot;s&quot;)) + 1)
            f = e.appendChild(dom.createElement(&quot;f&quot;))
            f.setAttribute(&quot;ii&quot;, i)
            f.setAttribute(&quot;t&quot;, j)
        }

    }
    WScript.Echo(&quot;1.ADD         :&quot; + time(t) + lNode(dom))
    if (MaxNode &lt; 80000) {
        dom = loadDom()
        r = dom.documentElement
        t = new Date()

        for (var j = 0; j &lt; MaxNode * 4; j++) {
            i = Math.round(Math.random() * MaxNode)
            var sPath = &quot;/root/e[@i=&#039;&quot; + i + &quot;&#039;]&quot; + &quot;[@k=&#039;&quot; + i % 3 + &quot;&#039;]&quot; + &quot;[@l=&#039;&quot; + i % 7 + &quot;&#039;]&quot;
            var e = dom.selectSingleNode(sPath)
            if (e == null) {
                e = r.appendChild(dom.createElement(&quot;e&quot;))
                e.setAttribute(&quot;i&quot;, i)
                e.setAttribute(&quot;k&quot;, i % 3)
                e.setAttribute(&quot;l&quot;, i % 7)
                e.setAttribute(&quot;s&quot;, &quot;0&quot;)
            }
            e.setAttribute(&quot;s&quot;, (1 * e.getAttribute(&quot;s&quot;)) + 1)
            f = e.appendChild(dom.createElement(&quot;f&quot;))
            f.setAttribute(&quot;ii&quot;, i)
            f.setAttribute(&quot;t&quot;, j)
        }
        WScript.Echo(&quot;2.selectSingle:&quot; + time(t) + lNode(dom))
    }
    dom = loadDom()
    r = dom.documentElement
    t = new Date()
    var o_i = {}

    for (var j = 0; j &lt; MaxNode * 4; j++) {
        i = Math.round(Math.random() * MaxNode)
        e = null
        var e_num = -1
        var x = [i, i % 3, i % 7].join(&quot;|&quot;)
        if (o_i[x] == null) {
            e = r.appendChild(dom.createElement(&quot;e&quot;))
            e.setAttribute(&quot;i&quot;, i)
            e.setAttribute(&quot;k&quot;, i % 3)
            e.setAttribute(&quot;l&quot;, i % 7)
            e.setAttribute(&quot;s&quot;, &quot;0&quot;)
        } else {
            e_num = o_i[x]
            e = r.childNodes[o_i[x]]
        }

        e.setAttribute(&quot;s&quot;, (1 * e.getAttribute(&quot;s&quot;)) + 1)
        f = e.appendChild(dom.createElement(&quot;f&quot;))
        f.setAttribute(&quot;ii&quot;, i)
        f.setAttribute(&quot;t&quot;, j)

        o_i[[e.getAttribute(&quot;i&quot;), e.getAttribute(&quot;k&quot;), e.getAttribute(&quot;l&quot;)].join(&quot;|&quot;)] = e_num &lt; 0 ? r.childNodes.length - 1 : e_num
    }
    WScript.Echo(&quot;3.selectObject:&quot; + time(t) + lNode(dom))

    t = new Date()
    var o_i = {}
    var sPath = &quot;//e&quot;

    var oNodes = dom.selectNodes(sPath);

    for (var i = 0; i &lt; oNodes.length; i++) {
        o_i[[e.getAttribute(&quot;i&quot;), e.getAttribute(&quot;k&quot;), e.getAttribute(&quot;l&quot;)].join(&quot;|&quot;)] = i
    }
    WScript.Echo(&quot;4.Load  Object:&quot; + time(t))

    WScript.Echo(&quot;&quot;)
}

function loadDom() {
    var dom
    dom = new ActiveXObject(&quot;msxml2.DOMDocument&quot;);
    dom.async = false;
    dom.validateOnParse = false;
    dom.resolveExternals = false;
    dom.appendChild(dom.createProcessingInstruction(&quot;xml&quot;, &quot;version=&#039;1.0&#039; encoding=&#039;windows-1251&#039;&quot;))
    dom.appendChild(dom.createElement(&quot;root&quot;))
    dom.setProperty(&quot;SelectionLanguage&quot;, &quot;XPath&quot;)
    return dom
}

function time(t) {
    var tt = new Date()
    var d = (new Date(tt - t))
    d.setMinutes(d.getMinutes() + d.getTimezoneOffset())
    return (&quot;&quot; + r(d.getHours()) + &quot;:&quot; + r(d.getMinutes()) + &quot;:&quot; + r(d.getSeconds()) + &quot;.&quot; + d.getMilliseconds())

        function r(d) {
            var s = &quot;00&quot; + d
            return (s.substr(s.length - 2, 2))
        }
}

function lNode(dom) {
    var sPath = &quot;//e&quot;
    var oNodes = dom.selectNodes(sPath);
    return (&quot; узлов:&quot; + oNodes.length)
}</code></pre></div></div></div><div class="fancy_spoiler_switcher"><div class="fancy_spoiler_switcher_header"><strong>+</strong>&nbsp;Результаты теста</div><div class="fancy_spoiler"><p>Результаты теста</p><div class="quotebox"><blockquote><p>узлов:1000<br />1.ADD&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:00:00.47 узлов:1000<br />2.selectSingle:00:00:00.890 узлов:982<br />3.selectObject:00:00:00.141 узлов:988<br />4.Load&nbsp; Object:00:00:00.0</p><p>узлов:5000<br />1.ADD&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:00:00.265 узлов:5000<br />2.selectSingle:00:00:19.392 узлов:4907<br />3.selectObject:00:00:01.109 узлов:4902<br />4.Load&nbsp; Object:00:00:00.31</p><p>узлов:10000<br />1.ADD&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:00:00.516 узлов:10000<br />2.selectSingle:00:01:19.392 узлов:9806<br />3.selectObject:00:00:03.313 узлов:9831<br />4.Load&nbsp; Object:00:00:00.94</p><p>узлов:25000<br />1.ADD&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:00:01.468 узлов:25000<br />2.selectSingle:00:09:16.983 узлов:24590<br />3.selectObject:00:00:19.422 узлов:24583<br />4.Load&nbsp; Object:00:00:00.219</p><p>узлов:40000<br />1.ADD&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:00:02.657 узлов:40000<br />2.selectSingle:00:24:22.365 узлов:39251<br />3.selectObject:00:00:58.658 узлов:39271<br />4.Load&nbsp; Object:00:00:00.406</p><p>узлов:50000<br />1.ADD&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:00:03.453 узлов:50000<br />2.selectSingle:00:38:27.12 узлов:49079<br />3.selectObject:00:01:39.159 узлов:49091<br />4.Load&nbsp; Object:00:00:00.609</p><p>узлов:75000<br />1.ADD&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:00:06.157 узлов:75000<br />2.selectSingle:01:27:43.805 узлов:73651<br />3.selectObject:00:04:27.100 узлов:73616<br />4.Load&nbsp; Object:00:00:00.812</p><p>узлов:100000<br />1.ADD&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:00:09.438 узлов:100000<br />3.selectObject:00:09:00.608 узлов:98126<br />4.Load&nbsp; Object:00:00:01.172</p><p>узлов:150000<br />1.ADD&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:00:17.891 узлов:150000<br />3.selectObject:00:23:03.441 узлов:147339<br />4.Load&nbsp; Object:00:00:01.672</p><p>узлов:200000<br />1.ADD&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;:00:00:28.860 узлов:200000<br />3.selectObject:00:44:33.89 узлов:196309<br />4.Load&nbsp; Object:00:00:02.547</p></blockquote></div></div></div>]]></content>
			<author>
				<name><![CDATA[badik]]></name>
				<uri>https://forum.script-coding.com/profile.php?id=21834</uri>
			</author>
			<updated>2014-02-27T07:37:43Z</updated>
			<id>https://forum.script-coding.com/viewtopic.php?pid=80407#p80407</id>
		</entry>
</feed>
