Announcement

Collapse
No announcement yet.

Problem with index search

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Problem with index search



    --=====================_11338964==_.ALT
    Content-Type: text/plain; charset="us-ascii"; format=flowed

    Hi everyone,

    I'm having a problem with generating results from a simple index, and I
    can't for the life of me figure out what the problem might be. I'm sure
    it's something obvious, but before I tear out what's left of my hair,
    perhaps someone could offer some suggestions.

    I photographic negative database of about 140,000 entries, with a series of
    keyword fields. These are added to a keyword database of 400,000 entries
    and is indexed on keyword $ image_number. The search program looks for a
    submitted keyword, then outputs the appropriate thumbnail images.

    The problem is this: When searching for a term , ie: "network", that term
    will NOT be found if their are other keywords such as "networks", "network
    lab", etc. The longer terms are always found. The search terms are RTRIMed
    and appear to be passed to the search program properly.

    Ideas?

    Thanks

    John



    John A. Brebner


    --=====================_11338964==_.ALT--


    #2
    Problem with index search



    Well, you did not write what code you use, but I assume you try it with
    MvFIND. That won't work so simply for the task you mention. MvFIND only
    finds records where the search expression, depending on the switch EXACT,
    matches the index key either exactly (from the first to the last character),
    or partially (but starting with the first character). This is apparently not
    the case in the example you posted.

    There are different possibilities to solve the dilemma: the simplest, but
    also the slowest, is using the MvFILTER command instead. Another possibility
    is combining MvFIND and MvFILTER with a special technique described on my
    website in the resource section. Yet another approach is accessing the
    problem from the other side - creating an indexed database of keywords, with
    second field containing ID of all records containing it - you can then use
    MvFIND to get the list of records immediately.

    Ivo Truxa

    | http://miva.truxoft.com
    | Advanced Miva Merchant modules



    -----Original Message-----
    From: [email protected] [mailto:[email protected]] On Behalf
    Of John Brebner
    Sent: Friday, March 11, 2005 2:26 PM
    To: [email protected]
    Subject: [meu] Problem with index search

    Hi everyone,

    I'm having a problem with generating results from a simple index, and I
    can't for the life of me figure out what the problem might be. I'm sure
    it's something obvious, but before I tear out what's left of my hair,
    perhaps someone could offer some suggestions.

    I photographic negative database of about 140,000 entries, with a series of
    keyword fields. These are added to a keyword database of 400,000 entries
    and is indexed on keyword $ image_number. The search program looks for a
    submitted keyword, then outputs the appropriate thumbnail images.

    The problem is this: When searching for a term , ie: "network", that term
    will NOT be found if their are other keywords such as "networks", "network
    lab", etc. The longer terms are always found. The search terms are RTRIMed
    and appear to be passed to the search program properly.

    Ideas?

    Thanks

    John



    John A. Brebner



    Comment


      #3
      Problem with index search



      Hi John,

      If both the search expression and the index key value begin with 'network'
      (case sensitive), then certainly yes, it should work. If it is this case,
      then your index expression is probably incorrectly written, or the index
      file invalid, or you do not use the same database alias consistently (if at
      all). Posting your code creating the index could tell more.

      But if the index key contains the word somewhere in the middle of multiple
      keywords, then it will not work, of course.

      PS: please hit the "reply all" button when replying to posts from the list,
      otherwise the answer goes only privately to the author.

      Ivo
      http://mivo.truxoft.com


      -----Original Message-----
      From: John Brebner [mailto:[email protected]]


      Hi Ivo,

      Yes, I'm using MvFIND without EXACT set...should that not return all records
      containing "network", including "network", "network lab" etc?

      John

      At 02:45 PM 3/11/2005 +0100, you wrote:


      Well, you did not write what code you use, but I assume you try it
      with
      MvFIND. That won't work so simply for the task you mention. MvFIND
      only
      finds records where the search expression, depending on the switch
      EXACT,
      matches the index key either exactly (from the first to the last
      character),
      or partially (but starting with the first character). This is
      apparently not
      the case in the example you posted.

      There are different possibilities to solve the dilemma: the
      simplest, but
      also the slowest, is using the MvFILTER command instead. Another
      possibility
      is combining MvFIND and MvFILTER with a special technique described
      on my
      website in the resource section. Yet another approach is accessing
      the
      problem from the other side - creating an indexed database of
      keywords, with
      second field containing ID of all records containing it - you can
      then use
      MvFIND to get the list of records immediately.

      Ivo Truxa

      | http://miva.truxoft.com <http://miva.truxoft.com/>
      | Advanced Miva Merchant modules



      -----Original Message-----
      From: [email protected] [mailto:[email protected]]
      On Behalf
      Of John Brebner
      Sent: Friday, March 11, 2005 2:26 PM
      To: [email protected]
      Subject: [meu] Problem with index search

      Hi everyone,

      I'm having a problem with generating results from a simple index,
      and I
      can't for the life of me figure out what the problem might be. I'm
      sure
      it's something obvious, but before I tear out what's left of my
      hair,
      perhaps someone could offer some suggestions.

      I photographic negative database of about 140,000 entries, with a
      series of
      keyword fields. These are added to a keyword database of 400,000
      entries
      and is indexed on keyword $ image_number. The search program looks
      for a
      submitted keyword, then outputs the appropriate thumbnail images.

      The problem is this: When searching for a term , ie: "network", that
      term
      will NOT be found if their are other keywords such as "networks",
      "network
      lab", etc. The longer terms are always found. The search terms are
      RTRIMed
      and appear to be passed to the search program properly.

      Ideas?

      Thanks

      John



      John A. Brebner



      John A. Brebner

      Researching the Brebner/Bremner families in the North-east of Scotland and
      around the world at www.brebner.com <<A HREF ="http://www.brebner.com/ ">http://www.brebner.com/ </A>>



      Comment


        #4
        Problem with index search



        --=====================_19691574==_.ALT
        Content-Type: text/plain; charset="us-ascii"; format=flowed

        Hi Ivo,

        Here's the code used for creating the index. All keywords are in upper case
        and left trimmed. The imagekey.dbf has only three fields, "keyword",
        "negnumber" and "digital".

        <MvOPEN
        NAME="crckeyword_alias"
        DATABASE="imagekey.dbf"
        TYPE="xbase3"
        >
        <MvMAKEINDEX
        NAME="crckeyword_alias"
        INDEXFILE="crckeyword.mvx"
        EXPR="{keyword $ negnumber}"
        >
        <MvCLOSE NAME="crckeyword_alias">

        The search term, first_keyword is input by the user.

        <MvOPEN
        NAME="crckeyword_alias"
        DATABASE="imagekey.dbf"
        TYPE="xbase3"
        INDEXES="crckeyword.mvx"
        >

        <MvOPEN
        NAME="crcneg_alias"
        DATABASE="crcneg.dbf"
        TYPE="xbase3"
        INDEXES="crcneg_number.mvx"
        >
        <MvASSIGN NAME="keysearch_first" VALUE="{RTRIM(toupper(first_keyword))}">
        <MvFIND
        NAME="crckeyword_alias"
        VALUE="{keysearch_first}"
        >

        (rest snipped...)

        When you enter a search term that is NOT in the keyword index, then eof()
        is reached. If "network" is input, and the term is in the keyword database,
        then no results are shown. Term "networks", "network lab" etc. produce results.

        I hope this gives you enough information!

        John


        At 04:04 PM 3/11/2005 +0100, you wrote:
        >Hi John,
        >
        >If both the search expression and the index key value begin with 'network'
        >(case sensitive), then certainly yes, it should work. If it is this case,
        >then your index expression is probably incorrectly written, or the index
        >file invalid, or you do not use the same database alias consistently (if at
        >all). Posting your code creating the index could tell more.
        >
        >But if the index key contains the word somewhere in the middle of multiple
        >keywords, then it will not work, of course.
        >
        >PS: please hit the "reply all" button when replying to posts from the list,
        >otherwise the answer goes only privately to the author.
        >
        >Ivo
        >http://mivo.truxoft.com
        >
        >
        >-----Original Message-----
        >From: John Brebner [mailto:[email protected]]
        >
        >
        >Hi Ivo,
        >
        >Yes, I'm using MvFIND without EXACT set...should that not return all records
        >containing "network", including "network", "network lab" etc?
        >
        >John
        >
        >At 02:45 PM 3/11/2005 +0100, you wrote:
        >
        >
        > Well, you did not write what code you use, but I assume you try it
        >with
        > MvFIND. That won't work so simply for the task you mention. MvFIND
        >only
        > finds records where the search expression, depending on the switch
        >EXACT,
        > matches the index key either exactly (from the first to the last
        >character),
        > or partially (but starting with the first character). This is
        >apparently not
        > the case in the example you posted.
        >
        > There are different possibilities to solve the dilemma: the
        >simplest, but
        > also the slowest, is using the MvFILTER command instead. Another
        >possibility
        > is combining MvFIND and MvFILTER with a special technique described
        >on my
        > website in the resource section. Yet another approach is accessing
        >the
        > problem from the other side - creating an indexed database of
        >keywords, with
        > second field containing ID of all records containing it - you can
        >then use
        > MvFIND to get the list of records immediately.
        >
        > Ivo Truxa
        >
        > | http://miva.truxoft.com <http://miva.truxoft.com/>
        > | Advanced Miva Merchant modules
        >
        >
        >
        > -----Original Message-----
        > From: [email protected] [mailto:[email protected]]
        >On Behalf
        > Of John Brebner
        > Sent: Friday, March 11, 2005 2:26 PM
        > To: [email protected]
        > Subject: [meu] Problem with index search
        >
        > Hi everyone,
        >
        > I'm having a problem with generating results from a simple index,
        >and I
        > can't for the life of me figure out what the problem might be. I'm
        >sure
        > it's something obvious, but before I tear out what's left of my
        >hair,
        > perhaps someone could offer some suggestions.
        >
        > I photographic negative database of about 140,000 entries, with a
        >series of
        > keyword fields. These are added to a keyword database of 400,000
        >entries
        > and is indexed on keyword $ image_number. The search program looks
        >for a
        > submitted keyword, then outputs the appropriate thumbnail images.
        >
        > The problem is this: When searching for a term , ie: "network", that
        >term
        > will NOT be found if their are other keywords such as "networks",
        >"network
        > lab", etc. The longer terms are always found. The search terms are
        >RTRIMed
        > and appear to be passed to the search program properly.
        >
        > Ideas?
        >
        > Thanks
        >
        > John
        >
        >
        >
        > John A. Brebner
        >
        >
        >
        >John A. Brebner
        >
        >Researching the Brebner/Bremner families in the North-east of Scotland and
        >around the world at www.brebner.com <<A HREF ="http://www.brebner.com/ ">http://www.brebner.com/ </A>>
        >
        >
        >

        Comment


          #5
          Problem with index search



          Hi John

          Two things: Ivo mentioned this already, but you should definitely scope =
          your
          variables. This is probably NOT the reason for the error, but that's a =
          main
          cause for nasty bugs and even security issues in a program.=20

          Then try to change your MvFIND in this way:

          <MvFIND
          NAME=3D"crckeyword_alias"
          VALUE=3D"{keysearch_first}"
          EXACT=3D"RANGE">

          By default, MvFIND tries to use the exact term, if "RANGE" is not =
          explicitly
          indicated. If you have it in there, then it'll work as Ivo explained.

          Although the RANGE-flag exists for quite a long time already, it hasn't =
          made
          it yet into the html-docs, but is still buried in one of the readme's.=20

          Hope that helps,

          Markus


          -----Original Message-----
          From: [email protected] [mailto:[email protected]] On =
          Behalf
          Of John Brebner
          Sent: Freitag, 11. M=E4rz 2005 16:45
          To: Ivo Truxa
          Cc: [email protected]
          Subject: RE: [meu] Problem with index search

          Hi Ivo,

          Here's the code used for creating the index. All keywords are in upper =
          case
          and left trimmed. The imagekey.dbf has only three fields, "keyword",
          "negnumber" and "digital".

          <MvOPEN
          NAME=3D"crckeyword_alias"
          DATABASE=3D"imagekey.dbf"
          TYPE=3D"xbase3"
          >
          <MvMAKEINDEX
          NAME=3D"crckeyword_alias"
          INDEXFILE=3D"crckeyword.mvx"
          EXPR=3D"{keyword $ negnumber}"
          >
          <MvCLOSE NAME=3D"crckeyword_alias">

          The search term, first_keyword is input by the user.

          <MvOPEN
          NAME=3D"crckeyword_alias"
          DATABASE=3D"imagekey.dbf"
          TYPE=3D"xbase3"
          INDEXES=3D"crckeyword.mvx"
          >

          <MvOPEN
          NAME=3D"crcneg_alias"
          DATABASE=3D"crcneg.dbf"
          TYPE=3D"xbase3"
          INDEXES=3D"crcneg_number.mvx"
          >
          <MvASSIGN NAME=3D"keysearch_first" =
          VALUE=3D"{RTRIM(toupper(first_keyword))}">
          <MvFIND
          NAME=3D"crckeyword_alias"
          VALUE=3D"{keysearch_first}"
          >

          (rest snipped...)

          When you enter a search term that is NOT in the keyword index, then =
          eof() is
          reached. If "network" is input, and the term is in the keyword database,
          then no results are shown. Term "networks", "network lab" etc. produce
          results.

          I hope this gives you enough information!

          John


          At 04:04 PM 3/11/2005 +0100, you wrote:
          >Hi John,
          >
          >If both the search expression and the index key value begin with =
          'network'
          >(case sensitive), then certainly yes, it should work. If it is this=20
          >case, then your index expression is probably incorrectly written, or=20
          >the index file invalid, or you do not use the same database alias=20
          >consistently (if at all). Posting your code creating the index could =
          tell
          more.
          >
          >But if the index key contains the word somewhere in the middle of=20
          >multiple keywords, then it will not work, of course.
          >
          >PS: please hit the "reply all" button when replying to posts from the=20
          >list, otherwise the answer goes only privately to the author.
          >
          >Ivo
          >http://mivo.truxoft.com
          >
          >
          >-----Original Message-----
          >From: John Brebner [mailto:[email protected]]
          >
          >
          >Hi Ivo,
          >
          >Yes, I'm using MvFIND without EXACT set...should that not return all=20
          >records containing "network", including "network", "network lab" etc?
          >
          >John
          >
          >At 02:45 PM 3/11/2005 +0100, you wrote:
          >
          >
          > Well, you did not write what code you use, but I assume you=20
          >try it with
          > MvFIND. That won't work so simply for the task you mention.=20
          >MvFIND only
          > finds records where the search expression, depending on the=20
          >switch EXACT,
          > matches the index key either exactly (from the first to the=20
          >last character),
          > or partially (but starting with the first character). This is=20
          >apparently not
          > the case in the example you posted.
          >
          > There are different possibilities to solve the dilemma: the=20
          >simplest, but
          > also the slowest, is using the MvFILTER command instead.=20
          >Another possibility
          > is combining MvFIND and MvFILTER with a special technique=20
          >described on my
          > website in the resource section. Yet another approach is=20
          >accessing the
          > problem from the other side - creating an indexed database of=20
          >keywords, with
          > second field containing ID of all records containing it - you=20
          >can then use
          > MvFIND to get the list of records immediately.
          >
          > Ivo Truxa
          >
          > | http://miva.truxoft.com <http://miva.truxoft.com/>
          > | Advanced Miva Merchant modules
          >
          >
          >
          > -----Original Message-----
          > From: [email protected]=20
          >[mailto:[email protected]]
          >On Behalf
          > Of John Brebner
          > Sent: Friday, March 11, 2005 2:26 PM
          > To: [email protected]
          > Subject: [meu] Problem with index search
          >
          > Hi everyone,
          >
          > I'm having a problem with generating results from a simple=20
          >index, and I
          > can't for the life of me figure out what the problem might be. =

          >I'm sure
          > it's something obvious, but before I tear out what's left of=20
          >my hair,
          > perhaps someone could offer some suggestions.
          >
          > I photographic negative database of about 140,000 entries,=20
          >with a series of
          > keyword fields. These are added to a keyword database of=20
          >400,000 entries
          > and is indexed on keyword $ image_number. The search program=20
          >looks for a
          > submitted keyword, then outputs the appropriate thumbnail =
          images.
          >
          > The problem is this: When searching for a term , ie:=20
          >"network", that term
          > will NOT be found if their are other keywords such as=20
          >"networks", "network
          > lab", etc. The longer terms are always found. The search terms =

          >are RTRIMed
          > and appear to be passed to the search program properly.
          >
          > Ideas?
          >
          > Thanks
          >
          > John
          >
          >
          >
          > John A. Brebner
          >
          >
          >
          >John A. Brebner
          >
          >Researching the Brebner/Bremner families in the North-east of Scotland=20
          >and around the world at www.brebner.com <<A HREF ="http://www.brebner.com/ ">http://www.brebner.com/ </A>>
          >
          >
          >

          Comment


            #6
            Problem with index search



            --=====================_21887963==_.ALT
            Content-Type: text/plain; charset="iso-8859-1"; format=flowed
            Content-Transfer-Encoding: quoted-printable

            Hi Markus,

            Tried that EXACT=3D"RANGE" addition, and now the eof() is reached with no=20
            results for any search term. With regard to the variable scope, aren't the=
            =20
            variables global by default? Then it shouldn't make a difference if I=20
            replaced keysearch_first with g.keysearch_first?

            John


            At 05:02 PM 3/11/2005 +0100, you wrote:
            >Hi John
            >
            >Two things: Ivo mentioned this already, but you should definitely scope=
            your
            >variables. This is probably NOT the reason for the error, but that's a main
            >cause for nasty bugs and even security issues in a program.
            >
            >Then try to change your MvFIND in this way:
            >
            ><MvFIND
            > NAME=3D"crckeyword_alias"
            > VALUE=3D"{keysearch_first}"
            > EXACT=3D"RANGE">
            >
            >By default, MvFIND tries to use the exact term, if "RANGE" is not=
            explicitly
            >indicated. If you have it in there, then it'll work as Ivo explained.
            >
            >Although the RANGE-flag exists for quite a long time already, it hasn't=
            made
            >it yet into the html-docs, but is still buried in one of the readme's.
            >
            >Hope that helps,
            >
            >Markus
            >
            >
            >-----Original Message-----
            >From: [email protected] [mailto:[email protected]] On=
            Behalf
            >Of John Brebner
            >Sent: Freitag, 11. M=E4rz 2005 16:45
            >To: Ivo Truxa
            >Cc: [email protected]
            >Subject: RE: [meu] Problem with index search
            >
            >Hi Ivo,
            >
            >Here's the code used for creating the index. All keywords are in upper case
            >and left trimmed. The imagekey.dbf has only three fields, "keyword",
            >"negnumber" and "digital".
            >
            ><MvOPEN
            > NAME=3D"crckeyword_alias"
            > DATABASE=3D"imagekey.dbf"
            > TYPE=3D"xbase3"
            > >
            ><MvMAKEINDEX
            > NAME=3D"crckeyword_alias"
            > INDEXFILE=3D"crckeyword.mvx"
            > EXPR=3D"{keyword $ negnumber}"
            > >
            ><MvCLOSE NAME=3D"crckeyword_alias">
            >
            >The search term, first_keyword is input by the user.
            >
            ><MvOPEN
            > NAME=3D"crckeyword_alias"
            > DATABASE=3D"imagekey.dbf"
            > TYPE=3D"xbase3"
            > INDEXES=3D"crckeyword.mvx"
            > >
            >
            ><MvOPEN
            > NAME=3D"crcneg_alias"
            > DATABASE=3D"crcneg.dbf"
            > TYPE=3D"xbase3"
            > INDEXES=3D"crcneg_number.mvx"
            > >
            ><MvASSIGN NAME=3D"keysearch_first" =
            VALUE=3D"{RTRIM(toupper(first_keyword))}">
            ><MvFIND
            > NAME=3D"crckeyword_alias"
            > VALUE=3D"{keysearch_first}"
            > >
            >
            >(rest snipped...)
            >
            >When you enter a search term that is NOT in the keyword index, then eof()=
            is
            >reached. If "network" is input, and the term is in the keyword database,
            >then no results are shown. Term "networks", "network lab" etc. produce
            >results.
            >
            >I hope this gives you enough information!
            >
            >John
            >
            >
            >At 04:04 PM 3/11/2005 +0100, you wrote:
            > >Hi John,
            > >
            > >If both the search expression and the index key value begin with=
            'network'
            > >(case sensitive), then certainly yes, it should work. If it is this
            > >case, then your index expression is probably incorrectly written, or
            > >the index file invalid, or you do not use the same database alias
            > >consistently (if at all). Posting your code creating the index could tell
            >more.
            > >
            > >But if the index key contains the word somewhere in the middle of
            > >multiple keywords, then it will not work, of course.
            > >
            > >PS: please hit the "reply all" button when replying to posts from the
            > >list, otherwise the answer goes only privately to the author.
            > >
            > >Ivo
            > >http://mivo.truxoft.com
            > >
            > >
            > >-----Original Message-----
            > >From: John Brebner [mailto:[email protected]]
            > >
            > >
            > >Hi Ivo,
            > >
            > >Yes, I'm using MvFIND without EXACT set...should that not return all
            > >records containing "network", including "network", "network lab" etc?
            > >
            > >John
            > >
            > >At 02:45 PM 3/11/2005 +0100, you wrote:
            > >
            > >
            > > Well, you did not write what code you use, but I assume you
            > >try it with
            > > MvFIND. That won't work so simply for the task you mention.
            > >MvFIND only
            > > finds records where the search expression, depending on the
            > >switch EXACT,
            > > matches the index key either exactly (from the first to the
            > >last character),
            > > or partially (but starting with the first character). This is
            > >apparently not
            > > the case in the example you posted.
            > >
            > > There are different possibilities to solve the dilemma: the
            > >simplest, but
            > > also the slowest, is using the MvFILTER command instead.
            > >Another possibility
            > > is combining MvFIND and MvFILTER with a special technique
            > >described on my
            > > website in the resource section. Yet another approach is
            > >accessing the
            > > problem from the other side - creating an indexed database of
            > >keywords, with
            > > second field containing ID of all records containing it - you
            > >can then use
            > > MvFIND to get the list of records immediately.
            > >
            > > Ivo Truxa
            > >
            > > | http://miva.truxoft.com <http://miva.truxoft.com/>
            > > | Advanced Miva Merchant modules
            > >
            > >
            > >
            > > -----Original Message-----
            > > From: [email protected]
            > >[mailto:[email protected]]
            > >On Behalf
            > > Of John Brebner
            > > Sent: Friday, March 11, 2005 2:26 PM
            > > To: [email protected]
            > > Subject: [meu] Problem with index search
            > >
            > > Hi everyone,
            > >
            > > I'm having a problem with generating results from a simple
            > >index, and I
            > > can't for the life of me figure out what the problem might be.
            > >I'm sure
            > > it's something obvious, but before I tear out what's left of
            > >my hair,
            > > perhaps someone could offer some suggestions.
            > >
            > > I photographic negative database of about 140,000 entries,
            > >with a series of
            > > keyword fields. These are added to a keyword database of
            > >400,000 entries
            > > and is indexed on keyword $ image_number. The search program
            > >looks for a
            > > submitted keyword, then outputs the appropriate thumbnail=
            images.
            > >
            > > The problem is this: When searching for a term , ie:
            > >"network", that term
            > > will NOT be found if their are other keywords such as
            > >"networks", "network
            > > lab", etc. The longer terms are always found. The search terms
            > >are RTRIMed
            > > and appear to be passed to the search program properly.
            > >
            > > Ideas?
            > >
            > > Thanks
            > >
            > > John
            > >
            > >
            > >
            > > John A. Brebner
            > >
            > >
            > >
            > >John A. Brebner
            > >
            > >Researching the Brebner/Bremner families in the North-east of Scotland
            > >and around the world at www.brebner.com <<A HREF ="http://www.brebner.com/ ">http://www.brebner.com/ </A>>
            > >
            > >
            > >

            Comment


              #7
              Problem with index search



              Hi John,

              About scoping: It is more a question of "best practice". An unscoped
              variable does not immediately default to global. Instead, the system =
              will
              first look if it can find a value for var among the system variables
              (s.var), then in the database d.var, then a global g.var and finally as =
              a
              local l.var.

              So if you have a database column called ID and the record cursor sits at =
              a
              record with the d.id =3D 25, but then you do an MvFIND for an unscoped =
              id (for
              example you search id=3D12), it can happen that Empresa will use d.id =
              =3D 25,
              because it has a higher priority.=20

              It is correct that variables passed through the command line are global =
              by
              default. Obviously, in the case of your variable "keysearch_first" there =
              is
              no ambiguity because there aren't any system vars with that name and for =
              a
              database variable the name is too long.

              However, as soon as you have lots of unscoped variables floating around =
              in
              your script, errors happen very quickly. It really doesn't cost much to
              write but a lot to debug, if you don't use them. Aside from the security
              aspect, this might even speed up your program (a few microseconds...).


              About the MvFIND: You should also rtrim(toupper(g.searchterm)) the
              searchterm in the MvFIND.=20

              NAME=3D"crckeyword_alias"
              VALUE=3D"{ RTRIM(toupper(keysearch_first))}"
              EXACT=3D"RANGE">


              Indices are case-sensitive. So maybe that's the reason why you don't get =
              any
              matches.=20

              Markus




              =20

              -----Original Message-----
              From: [email protected] [mailto:[email protected]] On =
              Behalf
              Of John Brebner
              Sent: Freitag, 11. M=E4rz 2005 17:21
              To: MvMarkus
              Cc: [email protected]
              Subject: RE: [meu] Problem with index search

              Hi Markus,

              Tried that EXACT=3D"RANGE" addition, and now the eof() is reached with =
              no
              results for any search term. With regard to the variable scope, aren't =
              the
              variables global by default? Then it shouldn't make a difference if I
              replaced keysearch_first with g.keysearch_first?

              John


              At 05:02 PM 3/11/2005 +0100, you wrote:
              >Hi John
              >
              >Two things: Ivo mentioned this already, but you should definitely scope =

              >your variables. This is probably NOT the reason for the error, but=20
              >that's a main cause for nasty bugs and even security issues in a =
              program.
              >
              >Then try to change your MvFIND in this way:
              >
              ><MvFIND
              > NAME=3D"crckeyword_alias"
              > VALUE=3D"{keysearch_first}"
              > EXACT=3D"RANGE">
              >
              >By default, MvFIND tries to use the exact term, if "RANGE" is not=20
              >explicitly indicated. If you have it in there, then it'll work as Ivo
              explained.
              >
              >Although the RANGE-flag exists for quite a long time already, it hasn't =

              >made it yet into the html-docs, but is still buried in one of the =
              readme's.
              >
              >Hope that helps,
              >
              >Markus
              >
              >
              >-----Original Message-----
              >From: [email protected] [mailto:[email protected]] On=20
              >Behalf Of John Brebner
              >Sent: Freitag, 11. M=E4rz 2005 16:45
              >To: Ivo Truxa
              >Cc: [email protected]
              >Subject: RE: [meu] Problem with index search
              >
              >Hi Ivo,
              >
              >Here's the code used for creating the index. All keywords are in upper=20
              >case and left trimmed. The imagekey.dbf has only three fields,=20
              >"keyword", "negnumber" and "digital".
              >
              ><MvOPEN
              > NAME=3D"crckeyword_alias"
              > DATABASE=3D"imagekey.dbf"
              > TYPE=3D"xbase3"
              > >
              ><MvMAKEINDEX
              > NAME=3D"crckeyword_alias"
              > INDEXFILE=3D"crckeyword.mvx"
              > EXPR=3D"{keyword $ negnumber}"
              > >
              ><MvCLOSE NAME=3D"crckeyword_alias">
              >
              >The search term, first_keyword is input by the user.
              >
              ><MvOPEN
              > NAME=3D"crckeyword_alias"
              > DATABASE=3D"imagekey.dbf"
              > TYPE=3D"xbase3"
              > INDEXES=3D"crckeyword.mvx"
              > >
              >
              ><MvOPEN
              > NAME=3D"crcneg_alias"
              > DATABASE=3D"crcneg.dbf"
              > TYPE=3D"xbase3"
              > INDEXES=3D"crcneg_number.mvx"
              > >
              ><MvASSIGN NAME=3D"keysearch_first" =20
              >VALUE=3D"{RTRIM(toupper(first_keyword))}">
              ><MvFIND
              > NAME=3D"crckeyword_alias"
              > VALUE=3D"{keysearch_first}"
              > >
              >
              >(rest snipped...)
              >
              >When you enter a search term that is NOT in the keyword index, then=20
              >eof() is reached. If "network" is input, and the term is in the keyword =

              >database, then no results are shown. Term "networks", "network lab"=20
              >etc. produce results.
              >
              >I hope this gives you enough information!
              >
              >John
              >
              >
              >At 04:04 PM 3/11/2005 +0100, you wrote:
              > >Hi John,
              > >
              > >If both the search expression and the index key value begin with
              'network'
              > >(case sensitive), then certainly yes, it should work. If it is this=20
              > >case, then your index expression is probably incorrectly written, or=20
              > >the index file invalid, or you do not use the same database alias=20
              > >consistently (if at all). Posting your code creating the index could=20
              > >tell
              >more.
              > >
              > >But if the index key contains the word somewhere in the middle of=20
              > >multiple keywords, then it will not work, of course.
              > >
              > >PS: please hit the "reply all" button when replying to posts from the =

              > >list, otherwise the answer goes only privately to the author.
              > >
              > >Ivo
              > >http://mivo.truxoft.com
              > >
              > >
              > >-----Original Message-----
              > >From: John Brebner [mailto:[email protected]]
              > >
              > >
              > >Hi Ivo,
              > >
              > >Yes, I'm using MvFIND without EXACT set...should that not return all=20
              > >records containing "network", including "network", "network lab" etc?
              > >
              > >John
              > >
              > >At 02:45 PM 3/11/2005 +0100, you wrote:
              > >
              > >
              > > Well, you did not write what code you use, but I assume you=20
              > >try it with
              > > MvFIND. That won't work so simply for the task you mention.
              > >MvFIND only
              > > finds records where the search expression, depending on the=20
              > >switch EXACT,
              > > matches the index key either exactly (from the first to the=20
              > >last character),
              > > or partially (but starting with the first character). This=20
              > >is apparently not
              > > the case in the example you posted.
              > >
              > > There are different possibilities to solve the dilemma: the=20
              > >simplest, but
              > > also the slowest, is using the MvFILTER command instead.
              > >Another possibility
              > > is combining MvFIND and MvFILTER with a special technique=20
              > >described on my
              > > website in the resource section. Yet another approach is=20
              > >accessing the
              > > problem from the other side - creating an indexed database=20
              > >of keywords, with
              > > second field containing ID of all records containing it -=20
              > >you can then use
              > > MvFIND to get the list of records immediately.
              > >
              > > Ivo Truxa
              > >
              > > | http://miva.truxoft.com <http://miva.truxoft.com/>
              > > | Advanced Miva Merchant modules
              > >
              > >
              > >
              > > -----Original Message-----
              > > From: [email protected]=20
              > >[mailto:[email protected]]
              > >On Behalf
              > > Of John Brebner
              > > Sent: Friday, March 11, 2005 2:26 PM
              > > To: [email protected]
              > > Subject: [meu] Problem with index search
              > >
              > > Hi everyone,
              > >
              > > I'm having a problem with generating results from a simple=20
              > >index, and I
              > > can't for the life of me figure out what the problem might =
              be.
              > >I'm sure
              > > it's something obvious, but before I tear out what's left of =

              > >my hair,
              > > perhaps someone could offer some suggestions.
              > >
              > > I photographic negative database of about 140,000 entries,=20
              > >with a series of
              > > keyword fields. These are added to a keyword database of=20
              > >400,000 entries
              > > and is indexed on keyword $ image_number. The search program =

              > >looks for a
              > > submitted keyword, then outputs the appropriate thumbnail
              images.
              > >
              > > The problem is this: When searching for a term , ie:
              > >"network", that term
              > > will NOT be found if their are other keywords such as=20
              > >"networks", "network
              > > lab", etc. The longer terms are always found. The search=20
              > >terms are RTRIMed
              > > and appear to be passed to the search program properly.
              > >
              > > Ideas?
              > >
              > > Thanks
              > >
              > > John
              > >
              > >
              > >
              > > John A. Brebner
              > >
              > >
              > >
              > >John A. Brebner
              > >
              > >Researching the Brebner/Bremner families in the North-east of=20
              > >Scotland and around the world at www.brebner.com
              <<A HREF ="http://www.brebner.com/ ">http://www.brebner.com/ </A>>
              > >
              > >
              > >

              Comment


                #8
                Problem with index search



                The scope precedence is actually as follows:
                system >> local >> database >> global

                It means the local variable will be used as the last possibility if =
                there
                are other variables or database fields of the same name

                See more about it in the manual:
                <A HREF ="http://www.miva.com/docs/ScriptMan_html/02-Variables.html#467683">http://www.miva.com/docs/ScriptMan_html/02-Variables.html#467683</A>

                The option range was added in v4.04. MvFIND is by default working in
                non-exact mode, only if the option EXACT appears in the tag, the exact =
                match
                will be looked for.

                The reasons for a non-match may be diverse - as Markus told, using the =
                scope
                in both MvFIND _and_ the index expression are rather important to avoid
                confusion of variables with the same name but different scope. Also by =
                not
                using the scope at a database variable, the field name is automatically
                taken as a part of the last opened database. If you added some records =
                when
                having other databases open, it is quite likely that the index is =
                completely
                invalid. I would suggest deleting the index file and recreating it again
                with a properly formed index expression.

                Easy way to verify if index contains the desired values, is opening it
                Notepad. Although it contains binary garbage, the index key values are =
                well
                readable.

                As Markus told, other possible reasons are whitespace characters, and
                character case. Using ltrim(rtrim(uppercase(alias.d.field))) in the =
                index
                expression is definitely wise to do.=20

                Ivo
                http://mivo.truxoft.com=20


                -----Original Message-----
                From: MvMarkus

                Hi John,

                About scoping: It is more a question of "best practice". An unscoped
                variable does not immediately default to global. Instead, the system =
                will
                first look if it can find a value for var among the system variables
                (s.var), then in the database d.var, then a global g.var and finally as =
                a
                local l.var.

                So if you have a database column called ID and the record cursor sits at =
                a
                record with the d.id =3D 25, but then you do an MvFIND for an unscoped =
                id (for
                example you search id=3D12), it can happen that Empresa will use d.id =
                =3D 25,
                because it has a higher priority.=20

                It is correct that variables passed through the command line are global =
                by
                default. Obviously, in the case of your variable "keysearch_first" there =
                is
                no ambiguity because there aren't any system vars with that name and for =
                a
                database variable the name is too long.

                However, as soon as you have lots of unscoped variables floating around =
                in
                your script, errors happen very quickly. It really doesn't cost much to
                write but a lot to debug, if you don't use them. Aside from the security
                aspect, this might even speed up your program (a few microseconds...).


                About the MvFIND: You should also rtrim(toupper(g.searchterm)) the
                searchterm in the MvFIND.=20

                NAME=3D"crckeyword_alias"
                VALUE=3D"{ RTRIM(toupper(keysearch_first))}"
                EXACT=3D"RANGE">


                Indices are case-sensitive. So maybe that's the reason why you don't get =
                any
                matches.=20

                Markus




                =20

                -----Original Message-----
                From: [email protected] [mailto:[email protected]] On =
                Behalf
                Of John Brebner
                Sent: Freitag, 11. M=E4rz 2005 17:21
                To: MvMarkus
                Cc: [email protected]
                Subject: RE: [meu] Problem with index search

                Hi Markus,

                Tried that EXACT=3D"RANGE" addition, and now the eof() is reached with =
                no
                results for any search term. With regard to the variable scope, aren't =
                the
                variables global by default? Then it shouldn't make a difference if I
                replaced keysearch_first with g.keysearch_first?

                John


                At 05:02 PM 3/11/2005 +0100, you wrote:
                >Hi John
                >
                >Two things: Ivo mentioned this already, but you should definitely scope =

                >your variables. This is probably NOT the reason for the error, but=20
                >that's a main cause for nasty bugs and even security issues in a =
                program.
                >
                >Then try to change your MvFIND in this way:
                >
                ><MvFIND
                > NAME=3D"crckeyword_alias"
                > VALUE=3D"{keysearch_first}"
                > EXACT=3D"RANGE">
                >
                >By default, MvFIND tries to use the exact term, if "RANGE" is not=20
                >explicitly indicated. If you have it in there, then it'll work as Ivo
                explained.
                >
                >Although the RANGE-flag exists for quite a long time already, it hasn't =

                >made it yet into the html-docs, but is still buried in one of the =
                readme's.
                >
                >Hope that helps,
                >
                >Markus
                >
                >
                >-----Original Message-----
                >From: [email protected] [mailto:[email protected]] On=20
                >Behalf Of John Brebner
                >Sent: Freitag, 11. M=E4rz 2005 16:45
                >To: Ivo Truxa
                >Cc: [email protected]
                >Subject: RE: [meu] Problem with index search
                >
                >Hi Ivo,
                >
                >Here's the code used for creating the index. All keywords are in upper=20
                >case and left trimmed. The imagekey.dbf has only three fields,=20
                >"keyword", "negnumber" and "digital".
                >
                ><MvOPEN
                > NAME=3D"crckeyword_alias"
                > DATABASE=3D"imagekey.dbf"
                > TYPE=3D"xbase3"
                > >
                ><MvMAKEINDEX
                > NAME=3D"crckeyword_alias"
                > INDEXFILE=3D"crckeyword.mvx"
                > EXPR=3D"{keyword $ negnumber}"
                > >
                ><MvCLOSE NAME=3D"crckeyword_alias">
                >
                >The search term, first_keyword is input by the user.
                >
                ><MvOPEN
                > NAME=3D"crckeyword_alias"
                > DATABASE=3D"imagekey.dbf"
                > TYPE=3D"xbase3"
                > INDEXES=3D"crckeyword.mvx"
                > >
                >
                ><MvOPEN
                > NAME=3D"crcneg_alias"
                > DATABASE=3D"crcneg.dbf"
                > TYPE=3D"xbase3"
                > INDEXES=3D"crcneg_number.mvx"
                > >
                ><MvASSIGN NAME=3D"keysearch_first" =20
                >VALUE=3D"{RTRIM(toupper(first_keyword))}">
                ><MvFIND
                > NAME=3D"crckeyword_alias"
                > VALUE=3D"{keysearch_first}"
                > >
                >
                >(rest snipped...)
                >
                >When you enter a search term that is NOT in the keyword index, then=20
                >eof() is reached. If "network" is input, and the term is in the keyword =

                >database, then no results are shown. Term "networks", "network lab"=20
                >etc. produce results.
                >
                >I hope this gives you enough information!
                >
                >John
                >
                >
                >At 04:04 PM 3/11/2005 +0100, you wrote:
                > >Hi John,
                > >
                > >If both the search expression and the index key value begin with
                'network'
                > >(case sensitive), then certainly yes, it should work. If it is this=20
                > >case, then your index expression is probably incorrectly written, or=20
                > >the index file invalid, or you do not use the same database alias=20
                > >consistently (if at all). Posting your code creating the index could=20
                > >tell
                >more.
                > >
                > >But if the index key contains the word somewhere in the middle of=20
                > >multiple keywords, then it will not work, of course.
                > >
                > >PS: please hit the "reply all" button when replying to posts from the =

                > >list, otherwise the answer goes only privately to the author.
                > >
                > >Ivo
                > >http://mivo.truxoft.com
                > >
                > >
                > >-----Original Message-----
                > >From: John Brebner [mailto:[email protected]]
                > >
                > >
                > >Hi Ivo,
                > >
                > >Yes, I'm using MvFIND without EXACT set...should that not return all=20
                > >records containing "network", including "network", "network lab" etc?
                > >
                > >John
                > >
                > >At 02:45 PM 3/11/2005 +0100, you wrote:
                > >
                > >
                > > Well, you did not write what code you use, but I assume you=20
                > >try it with
                > > MvFIND. That won't work so simply for the task you mention.
                > >MvFIND only
                > > finds records where the search expression, depending on the=20
                > >switch EXACT,
                > > matches the index key either exactly (from the first to the=20
                > >last character),
                > > or partially (but starting with the first character). This=20
                > >is apparently not
                > > the case in the example you posted.
                > >
                > > There are different possibilities to solve the dilemma: the=20
                > >simplest, but
                > > also the slowest, is using the MvFILTER command instead.
                > >Another possibility
                > > is combining MvFIND and MvFILTER with a special technique=20
                > >described on my
                > > website in the resource section. Yet another approach is=20
                > >accessing the
                > > problem from the other side - creating an indexed database=20
                > >of keywords, with
                > > second field containing ID of all records containing it -=20
                > >you can then use
                > > MvFIND to get the list of records immediately.
                > >
                > > Ivo Truxa
                > >
                > > | http://miva.truxoft.com <http://miva.truxoft.com/>
                > > | Advanced Miva Merchant modules
                > >
                > >
                > >
                > > -----Original Message-----
                > > From: [email protected]=20
                > >[mailto:[email protected]]
                > >On Behalf
                > > Of John Brebner
                > > Sent: Friday, March 11, 2005 2:26 PM
                > > To: [email protected]
                > > Subject: [meu] Problem with index search
                > >
                > > Hi everyone,
                > >
                > > I'm having a problem with generating results from a simple=20
                > >index, and I
                > > can't for the life of me figure out what the problem might =
                be.
                > >I'm sure
                > > it's something obvious, but before I tear out what's left of =

                > >my hair,
                > > perhaps someone could offer some suggestions.
                > >
                > > I photographic negative database of about 140,000 entries,=20
                > >with a series of
                > > keyword fields. These are added to a keyword database of=20
                > >400,000 entries
                > > and is indexed on keyword $ image_number. The search program =

                > >looks for a
                > > submitted keyword, then outputs the appropriate thumbnail
                images.
                > >
                > > The problem is this: When searching for a term , ie:
                > >"network", that term
                > > will NOT be found if their are other keywords such as=20
                > >"networks", "network
                > > lab", etc. The longer terms are always found. The search=20
                > >terms are RTRIMed
                > > and appear to be passed to the search program properly.
                > >
                > > Ideas?
                > >
                > > Thanks
                > >
                > > John
                > >
                > >
                > >
                > > John A. Brebner
                > >
                > >
                > >
                > >John A. Brebner
                > >
                > >Researching the Brebner/Bremner families in the North-east of=20
                > >Scotland and around the world at www.brebner.com
                <<A HREF ="http://www.brebner.com/ ">http://www.brebner.com/ </A>>


                Comment


                  #9
                  Problem with index search



                  Erratum:
                  "... the local variable will be used as the last ..."
                  Should have been "... the global ..." of course

                  Ivo Truxa

                  | http://miva.truxoft.com
                  | Advanced Miva Merchant modules



                  -----Original Message-----
                  From: Ivo Truxa


                  The scope precedence is actually as follows:
                  system >> local >> database >> global

                  It means the local variable will be used as the last possibility if =
                  there
                  are other variables or database fields of the same name

                  See more about it in the manual:
                  <A HREF ="http://www.miva.com/docs/ScriptMan_html/02-Variables.html#467683">http://www.miva.com/docs/ScriptMan_html/02-Variables.html#467683</A>

                  The option range was added in v4.04. MvFIND is by default working in
                  non-exact mode, only if the option EXACT appears in the tag, the exact =
                  match
                  will be looked for.

                  The reasons for a non-match may be diverse - as Markus told, using the =
                  scope
                  in both MvFIND _and_ the index expression are rather important to avoid
                  confusion of variables with the same name but different scope. Also by =
                  not
                  using the scope at a database variable, the field name is automatically
                  taken as a part of the last opened database. If you added some records =
                  when
                  having other databases open, it is quite likely that the index is =
                  completely
                  invalid. I would suggest deleting the index file and recreating it again
                  with a properly formed index expression.

                  Easy way to verify if index contains the desired values, is opening it
                  Notepad. Although it contains binary garbage, the index key values are =
                  well
                  readable.

                  As Markus told, other possible reasons are whitespace characters, and
                  character case. Using ltrim(rtrim(uppercase(alias.d.field))) in the =
                  index
                  expression is definitely wise to do.=20

                  Ivo
                  http://mivo.truxoft.com=20


                  -----Original Message-----
                  From: MvMarkus

                  Hi John,

                  About scoping: It is more a question of "best practice". An unscoped
                  variable does not immediately default to global. Instead, the system =
                  will
                  first look if it can find a value for var among the system variables
                  (s.var), then in the database d.var, then a global g.var and finally as =
                  a
                  local l.var.

                  So if you have a database column called ID and the record cursor sits at =
                  a
                  record with the d.id =3D 25, but then you do an MvFIND for an unscoped =
                  id (for
                  example you search id=3D12), it can happen that Empresa will use d.id =
                  =3D 25,
                  because it has a higher priority.=20

                  It is correct that variables passed through the command line are global =
                  by
                  default. Obviously, in the case of your variable "keysearch_first" there =
                  is
                  no ambiguity because there aren't any system vars with that name and for =
                  a
                  database variable the name is too long.

                  However, as soon as you have lots of unscoped variables floating around =
                  in
                  your script, errors happen very quickly. It really doesn't cost much to
                  write but a lot to debug, if you don't use them. Aside from the security
                  aspect, this might even speed up your program (a few microseconds...).


                  About the MvFIND: You should also rtrim(toupper(g.searchterm)) the
                  searchterm in the MvFIND.=20

                  NAME=3D"crckeyword_alias"
                  VALUE=3D"{ RTRIM(toupper(keysearch_first))}"
                  EXACT=3D"RANGE">


                  Indices are case-sensitive. So maybe that's the reason why you don't get =
                  any
                  matches.=20

                  Markus




                  =20

                  -----Original Message-----
                  From: [email protected] [mailto:[email protected]] On =
                  Behalf
                  Of John Brebner
                  Sent: Freitag, 11. M=E4rz 2005 17:21
                  To: MvMarkus
                  Cc: [email protected]
                  Subject: RE: [meu] Problem with index search

                  Hi Markus,

                  Tried that EXACT=3D"RANGE" addition, and now the eof() is reached with =
                  no
                  results for any search term. With regard to the variable scope, aren't =
                  the
                  variables global by default? Then it shouldn't make a difference if I
                  replaced keysearch_first with g.keysearch_first?

                  John


                  At 05:02 PM 3/11/2005 +0100, you wrote:
                  >Hi John
                  >
                  >Two things: Ivo mentioned this already, but you should definitely scope =

                  >your variables. This is probably NOT the reason for the error, but=20
                  >that's a main cause for nasty bugs and even security issues in a =
                  program.
                  >
                  >Then try to change your MvFIND in this way:
                  >
                  ><MvFIND
                  > NAME=3D"crckeyword_alias"
                  > VALUE=3D"{keysearch_first}"
                  > EXACT=3D"RANGE">
                  >
                  >By default, MvFIND tries to use the exact term, if "RANGE" is not=20
                  >explicitly indicated. If you have it in there, then it'll work as Ivo
                  explained.
                  >
                  >Although the RANGE-flag exists for quite a long time already, it hasn't =

                  >made it yet into the html-docs, but is still buried in one of the =
                  readme's.
                  >
                  >Hope that helps,
                  >
                  >Markus
                  >
                  >
                  >-----Original Message-----
                  >From: [email protected] [mailto:[email protected]] On=20
                  >Behalf Of John Brebner
                  >Sent: Freitag, 11. M=E4rz 2005 16:45
                  >To: Ivo Truxa
                  >Cc: [email protected]
                  >Subject: RE: [meu] Problem with index search
                  >
                  >Hi Ivo,
                  >
                  >Here's the code used for creating the index. All keywords are in upper=20
                  >case and left trimmed. The imagekey.dbf has only three fields,=20
                  >"keyword", "negnumber" and "digital".
                  >
                  ><MvOPEN
                  > NAME=3D"crckeyword_alias"
                  > DATABASE=3D"imagekey.dbf"
                  > TYPE=3D"xbase3"
                  > >
                  ><MvMAKEINDEX
                  > NAME=3D"crckeyword_alias"
                  > INDEXFILE=3D"crckeyword.mvx"
                  > EXPR=3D"{keyword $ negnumber}"
                  > >
                  ><MvCLOSE NAME=3D"crckeyword_alias">
                  >
                  >The search term, first_keyword is input by the user.
                  >
                  ><MvOPEN
                  > NAME=3D"crckeyword_alias"
                  > DATABASE=3D"imagekey.dbf"
                  > TYPE=3D"xbase3"
                  > INDEXES=3D"crckeyword.mvx"
                  > >
                  >
                  ><MvOPEN
                  > NAME=3D"crcneg_alias"
                  > DATABASE=3D"crcneg.dbf"
                  > TYPE=3D"xbase3"
                  > INDEXES=3D"crcneg_number.mvx"
                  > >
                  ><MvASSIGN NAME=3D"keysearch_first" =20
                  >VALUE=3D"{RTRIM(toupper(first_keyword))}">
                  ><MvFIND
                  > NAME=3D"crckeyword_alias"
                  > VALUE=3D"{keysearch_first}"
                  > >
                  >
                  >(rest snipped...)
                  >
                  >When you enter a search term that is NOT in the keyword index, then=20
                  >eof() is reached. If "network" is input, and the term is in the keyword =

                  >database, then no results are shown. Term "networks", "network lab"=20
                  >etc. produce results.
                  >
                  >I hope this gives you enough information!
                  >
                  >John
                  >
                  >
                  >At 04:04 PM 3/11/2005 +0100, you wrote:
                  > >Hi John,
                  > >
                  > >If both the search expression and the index key value begin with
                  'network'
                  > >(case sensitive), then certainly yes, it should work. If it is this=20
                  > >case, then your index expression is probably incorrectly written, or=20
                  > >the index file invalid, or you do not use the same database alias=20
                  > >consistently (if at all). Posting your code creating the index could=20
                  > >tell
                  >more.
                  > >
                  > >But if the index key contains the word somewhere in the middle of=20
                  > >multiple keywords, then it will not work, of course.
                  > >
                  > >PS: please hit the "reply all" button when replying to posts from the =

                  > >list, otherwise the answer goes only privately to the author.
                  > >
                  > >Ivo
                  > >http://mivo.truxoft.com
                  > >
                  > >
                  > >-----Original Message-----
                  > >From: John Brebner [mailto:[email protected]]
                  > >
                  > >
                  > >Hi Ivo,
                  > >
                  > >Yes, I'm using MvFIND without EXACT set...should that not return all=20
                  > >records containing "network", including "network", "network lab" etc?
                  > >
                  > >John
                  > >
                  > >At 02:45 PM 3/11/2005 +0100, you wrote:
                  > >
                  > >
                  > > Well, you did not write what code you use, but I assume you=20
                  > >try it with
                  > > MvFIND. That won't work so simply for the task you mention.
                  > >MvFIND only
                  > > finds records where the search expression, depending on the=20
                  > >switch EXACT,
                  > > matches the index key either exactly (from the first to the=20
                  > >last character),
                  > > or partially (but starting with the first character). This=20
                  > >is apparently not
                  > > the case in the example you posted.
                  > >
                  > > There are different possibilities to solve the dilemma: the=20
                  > >simplest, but
                  > > also the slowest, is using the MvFILTER command instead.
                  > >Another possibility
                  > > is combining MvFIND and MvFILTER with a special technique=20
                  > >described on my
                  > > website in the resource section. Yet another approach is=20
                  > >accessing the
                  > > problem from the other side - creating an indexed database=20
                  > >of keywords, with
                  > > second field containing ID of all records containing it -=20
                  > >you can then use
                  > > MvFIND to get the list of records immediately.
                  > >
                  > > Ivo Truxa
                  > >
                  > > | http://miva.truxoft.com <http://miva.truxoft.com/>
                  > > | Advanced Miva Merchant modules
                  > >
                  > >
                  > >
                  > > -----Original Message-----
                  > > From: [email protected]=20
                  > >[mailto:[email protected]]
                  > >On Behalf
                  > > Of John Brebner
                  > > Sent: Friday, March 11, 2005 2:26 PM
                  > > To: [email protected]
                  > > Subject: [meu] Problem with index search
                  > >
                  > > Hi everyone,
                  > >
                  > > I'm having a problem with generating results from a simple=20
                  > >index, and I
                  > > can't for the life of me figure out what the problem might =
                  be.
                  > >I'm sure
                  > > it's something obvious, but before I tear out what's left of =

                  > >my hair,
                  > > perhaps someone could offer some suggestions.
                  > >
                  > > I photographic negative database of about 140,000 entries,=20
                  > >with a series of
                  > > keyword fields. These are added to a keyword database of=20
                  > >400,000 entries
                  > > and is indexed on keyword $ image_number. The search program =

                  > >looks for a
                  > > submitted keyword, then outputs the appropriate thumbnail
                  images.
                  > >
                  > > The problem is this: When searching for a term , ie:
                  > >"network", that term
                  > > will NOT be found if their are other keywords such as=20
                  > >"networks", "network
                  > > lab", etc. The longer terms are always found. The search=20
                  > >terms are RTRIMed
                  > > and appear to be passed to the search program properly.
                  > >
                  > > Ideas?
                  > >
                  > > Thanks
                  > >
                  > > John
                  > >
                  > >
                  > >


                  Comment


                    #10
                    Problem with index search



                    "The scope precedence is actually as follows:
                    system >> local >> database >> global"

                    Well - time for me to read the manual again.... Thanks Ivo!=20

                    Markus

                    BTW. Is your "uppercase()" in "ltrim(rtrim(uppercase(alias.d.field)))" a
                    typo or yet another undocumented built-in function? I've never heard of =
                    that
                    one so far (...and it doesn't seem to compile)



                    -----Original Message-----
                    From: [email protected] [mailto:[email protected]] On =
                    Behalf
                    Of Ivo Truxa
                    Sent: Freitag, 11. M=E4rz 2005 18:15
                    To: [email protected]
                    Subject: RE: [meu] Problem with index search

                    The scope precedence is actually as follows:
                    system >> local >> database >> global

                    It means the local variable will be used as the last possibility if =
                    there
                    are other variables or database fields of the same name

                    See more about it in the manual:
                    <A HREF ="http://www.miva.com/docs/ScriptMan_html/02-Variables.html#467683">http://www.miva.com/docs/ScriptMan_html/02-Variables.html#467683</A>

                    The option range was added in v4.04. MvFIND is by default working in
                    non-exact mode, only if the option EXACT appears in the tag, the exact =
                    match
                    will be looked for.

                    The reasons for a non-match may be diverse - as Markus told, using the =
                    scope
                    in both MvFIND _and_ the index expression are rather important to avoid
                    confusion of variables with the same name but different scope. Also by =
                    not
                    using the scope at a database variable, the field name is automatically
                    taken as a part of the last opened database. If you added some records =
                    when
                    having other databases open, it is quite likely that the index is =
                    completely
                    invalid. I would suggest deleting the index file and recreating it again
                    with a properly formed index expression.

                    Easy way to verify if index contains the desired values, is opening it
                    Notepad. Although it contains binary garbage, the index key values are =
                    well
                    readable.

                    As Markus told, other possible reasons are whitespace characters, and
                    character case. Using ltrim(rtrim(uppercase(alias.d.field))) in the =
                    index
                    expression is definitely wise to do.=20

                    Ivo
                    http://mivo.truxoft.com=20


                    -----Original Message-----
                    From: MvMarkus

                    Hi John,

                    About scoping: It is more a question of "best practice". An unscoped
                    variable does not immediately default to global. Instead, the system =
                    will
                    first look if it can find a value for var among the system variables
                    (s.var), then in the database d.var, then a global g.var and finally as =
                    a
                    local l.var.

                    So if you have a database column called ID and the record cursor sits at =
                    a
                    record with the d.id =3D 25, but then you do an MvFIND for an unscoped =
                    id (for
                    example you search id=3D12), it can happen that Empresa will use d.id =
                    =3D 25,
                    because it has a higher priority.=20

                    It is correct that variables passed through the command line are global =
                    by
                    default. Obviously, in the case of your variable "keysearch_first" there =
                    is
                    no ambiguity because there aren't any system vars with that name and for =
                    a
                    database variable the name is too long.

                    However, as soon as you have lots of unscoped variables floating around =
                    in
                    your script, errors happen very quickly. It really doesn't cost much to
                    write but a lot to debug, if you don't use them. Aside from the security
                    aspect, this might even speed up your program (a few microseconds...).


                    About the MvFIND: You should also rtrim(toupper(g.searchterm)) the
                    searchterm in the MvFIND.=20

                    NAME=3D"crckeyword_alias"
                    VALUE=3D"{ RTRIM(toupper(keysearch_first))}"
                    EXACT=3D"RANGE">


                    Indices are case-sensitive. So maybe that's the reason why you don't get =
                    any
                    matches.=20

                    Markus




                    =20

                    -----Original Message-----
                    From: [email protected] [mailto:[email protected]] On =
                    Behalf
                    Of John Brebner
                    Sent: Freitag, 11. M=E4rz 2005 17:21
                    To: MvMarkus
                    Cc: [email protected]
                    Subject: RE: [meu] Problem with index search

                    Hi Markus,

                    Tried that EXACT=3D"RANGE" addition, and now the eof() is reached with =
                    no
                    results for any search term. With regard to the variable scope, aren't =
                    the
                    variables global by default? Then it shouldn't make a difference if I
                    replaced keysearch_first with g.keysearch_first?

                    John


                    At 05:02 PM 3/11/2005 +0100, you wrote:
                    >Hi John
                    >
                    >Two things: Ivo mentioned this already, but you should definitely scope =

                    >your variables. This is probably NOT the reason for the error, but=20
                    >that's a main cause for nasty bugs and even security issues in a =
                    program.
                    >
                    >Then try to change your MvFIND in this way:
                    >
                    ><MvFIND
                    > NAME=3D"crckeyword_alias"
                    > VALUE=3D"{keysearch_first}"
                    > EXACT=3D"RANGE">
                    >
                    >By default, MvFIND tries to use the exact term, if "RANGE" is not=20
                    >explicitly indicated. If you have it in there, then it'll work as Ivo
                    explained.
                    >
                    >Although the RANGE-flag exists for quite a long time already, it hasn't =

                    >made it yet into the html-docs, but is still buried in one of the =
                    readme's.
                    >
                    >Hope that helps,
                    >
                    >Markus
                    >
                    >
                    >-----Original Message-----
                    >From: [email protected] [mailto:[email protected]] On=20
                    >Behalf Of John Brebner
                    >Sent: Freitag, 11. M=E4rz 2005 16:45
                    >To: Ivo Truxa
                    >Cc: [email protected]
                    >Subject: RE: [meu] Problem with index search
                    >
                    >Hi Ivo,
                    >
                    >Here's the code used for creating the index. All keywords are in upper=20
                    >case and left trimmed. The imagekey.dbf has only three fields,=20
                    >"keyword", "negnumber" and "digital".
                    >
                    ><MvOPEN
                    > NAME=3D"crckeyword_alias"
                    > DATABASE=3D"imagekey.dbf"
                    > TYPE=3D"xbase3"
                    > >
                    ><MvMAKEINDEX
                    > NAME=3D"crckeyword_alias"
                    > INDEXFILE=3D"crckeyword.mvx"
                    > EXPR=3D"{keyword $ negnumber}"
                    > >
                    ><MvCLOSE NAME=3D"crckeyword_alias">
                    >
                    >The search term, first_keyword is input by the user.
                    >
                    ><MvOPEN
                    > NAME=3D"crckeyword_alias"
                    > DATABASE=3D"imagekey.dbf"
                    > TYPE=3D"xbase3"
                    > INDEXES=3D"crckeyword.mvx"
                    > >
                    >
                    ><MvOPEN
                    > NAME=3D"crcneg_alias"
                    > DATABASE=3D"crcneg.dbf"
                    > TYPE=3D"xbase3"
                    > INDEXES=3D"crcneg_number.mvx"
                    > >
                    ><MvASSIGN NAME=3D"keysearch_first" =20
                    >VALUE=3D"{RTRIM(toupper(first_keyword))}">
                    ><MvFIND
                    > NAME=3D"crckeyword_alias"
                    > VALUE=3D"{keysearch_first}"
                    > >
                    >
                    >(rest snipped...)
                    >
                    >When you enter a search term that is NOT in the keyword index, then
                    >eof() is reached. If "network" is input, and the term is in the keyword =

                    >database, then no results are shown. Term "networks", "network lab"
                    >etc. produce results.
                    >
                    >I hope this gives you enough information!
                    >
                    >John
                    >
                    >
                    >At 04:04 PM 3/11/2005 +0100, you wrote:
                    > >Hi John,
                    > >
                    > >If both the search expression and the index key value begin with
                    'network'
                    > >(case sensitive), then certainly yes, it should work. If it is this=20
                    > >case, then your index expression is probably incorrectly written, or=20
                    > >the index file invalid, or you do not use the same database alias=20
                    > >consistently (if at all). Posting your code creating the index could=20
                    > >tell
                    >more.
                    > >
                    > >But if the index key contains the word somewhere in the middle of=20
                    > >multiple keywords, then it will not work, of course.
                    > >
                    > >PS: please hit the "reply all" button when replying to posts from the =

                    > >list, otherwise the answer goes only privately to the author.
                    > >
                    > >Ivo
                    > >http://mivo.truxoft.com
                    > >
                    > >
                    > >-----Original Message-----
                    > >From: John Brebner [mailto:[email protected]]
                    > >
                    > >
                    > >Hi Ivo,
                    > >
                    > >Yes, I'm using MvFIND without EXACT set...should that not return all=20
                    > >records containing "network", including "network", "network lab" etc?
                    > >
                    > >John
                    > >
                    > >At 02:45 PM 3/11/2005 +0100, you wrote:
                    > >
                    > >
                    > > Well, you did not write what code you use, but I assume you=20
                    > >try it with
                    > > MvFIND. That won't work so simply for the task you mention.
                    > >MvFIND only
                    > > finds records where the search expression, depending on the=20
                    > >switch EXACT,
                    > > matches the index key either exactly (from the first to the=20
                    > >last character),
                    > > or partially (but starting with the first character). This=20
                    > >is apparently not
                    > > the case in the example you posted.
                    > >
                    > > There are different possibilities to solve the dilemma: the=20
                    > >simplest, but
                    > > also the slowest, is using the MvFILTER command instead.
                    > >Another possibility
                    > > is combining MvFIND and MvFILTER with a special technique=20
                    > >described on my
                    > > website in the resource section. Yet another approach is=20
                    > >accessing the
                    > > problem from the other side - creating an indexed database=20
                    > >of keywords, with
                    > > second field containing ID of all records containing it -=20
                    > >you can then use
                    > > MvFIND to get the list of records immediately.
                    > >
                    > > Ivo Truxa
                    > >
                    > > | http://miva.truxoft.com <http://miva.truxoft.com/>
                    > > | Advanced Miva Merchant modules
                    > >
                    > >
                    > >
                    > > -----Original Message-----
                    > > From: [email protected]=20
                    > >[mailto:[email protected]]
                    > >On Behalf
                    > > Of John Brebner
                    > > Sent: Friday, March 11, 2005 2:26 PM
                    > > To: [email protected]
                    > > Subject: [meu] Problem with index search
                    > >
                    > > Hi everyone,
                    > >
                    > > I'm having a problem with generating results from a simple=20
                    > >index, and I
                    > > can't for the life of me figure out what the problem might =
                    be.
                    > >I'm sure
                    > > it's something obvious, but before I tear out what's left of =

                    > >my hair,
                    > > perhaps someone could offer some suggestions.
                    > >
                    > > I photographic negative database of about 140,000 entries,=20
                    > >with a series of
                    > > keyword fields. These are added to a keyword database of=20
                    > >400,000 entries
                    > > and is indexed on keyword $ image_number. The search program =

                    > >looks for a
                    > > submitted keyword, then outputs the appropriate thumbnail
                    images.
                    > >
                    > > The problem is this: When searching for a term , ie:
                    > >"network", that term
                    > > will NOT be found if their are other keywords such as=20
                    > >"networks", "network
                    > > lab", etc. The longer terms are always found. The search=20
                    > >terms are RTRIMed
                    > > and appear to be passed to the search program properly.
                    > >
                    > > Ideas?
                    > >
                    > > Thanks
                    > >
                    > > John
                    > >
                    > >
                    > >
                    > > John A. Brebner
                    > >
                    > >
                    > >
                    > >John A. Brebner
                    > >
                    > >Researching the Brebner/Bremner families in the North-east of=20
                    > >Scotland and around the world at www.brebner.com
                    <<A HREF ="http://www.brebner.com/ ">http://www.brebner.com/ </A>>


                    Comment


                      #11
                      Problem with index search



                      Yes, sorry, that was another slip (just thinking in another language).
                      toupper() is the right function in Miva Script, of course.

                      Thanks for catching it - score equalled :)
                      Ivo
                      http://mivo.truxoft.com=20




                      -----Original Message-----
                      From: MvMarkus [mailto:[email protected]]=20
                      Sent: Friday, March 11, 2005 6:29 PM
                      To: 'Ivo Truxa'; [email protected]
                      Subject: RE: [meu] Problem with index search

                      "The scope precedence is actually as follows:
                      system >> local >> database >> global"

                      Well - time for me to read the manual again.... Thanks Ivo!=20

                      Markus

                      BTW. Is your "uppercase()" in "ltrim(rtrim(uppercase(alias.d.field)))" a
                      typo or yet another undocumented built-in function? I've never heard of =
                      that
                      one so far (...and it doesn't seem to compile)



                      -----Original Message-----
                      From: [email protected] [mailto:[email protected]] On =
                      Behalf
                      Of Ivo Truxa
                      Sent: Freitag, 11. M=E4rz 2005 18:15
                      To: [email protected]
                      Subject: RE: [meu] Problem with index search

                      The scope precedence is actually as follows:
                      system >> local >> database >> global

                      It means the local variable will be used as the last possibility if =
                      there
                      are other variables or database fields of the same name

                      See more about it in the manual:
                      <A HREF ="http://www.miva.com/docs/ScriptMan_html/02-Variables.html#467683">http://www.miva.com/docs/ScriptMan_html/02-Variables.html#467683</A>

                      The option range was added in v4.04. MvFIND is by default working in
                      non-exact mode, only if the option EXACT appears in the tag, the exact =
                      match
                      will be looked for.

                      The reasons for a non-match may be diverse - as Markus told, using the =
                      scope
                      in both MvFIND _and_ the index expression are rather important to avoid
                      confusion of variables with the same name but different scope. Also by =
                      not
                      using the scope at a database variable, the field name is automatically
                      taken as a part of the last opened database. If you added some records =
                      when
                      having other databases open, it is quite likely that the index is =
                      completely
                      invalid. I would suggest deleting the index file and recreating it again
                      with a properly formed index expression.

                      Easy way to verify if index contains the desired values, is opening it
                      Notepad. Although it contains binary garbage, the index key values are =
                      well
                      readable.

                      As Markus told, other possible reasons are whitespace characters, and
                      character case. Using ltrim(rtrim(uppercase(alias.d.field))) in the =
                      index
                      expression is definitely wise to do.=20

                      Ivo
                      http://mivo.truxoft.com=20


                      -----Original Message-----
                      From: MvMarkus

                      Hi John,

                      About scoping: It is more a question of "best practice". An unscoped
                      variable does not immediately default to global. Instead, the system =
                      will
                      first look if it can find a value for var among the system variables
                      (s.var), then in the database d.var, then a global g.var and finally as =
                      a
                      local l.var.

                      So if you have a database column called ID and the record cursor sits at =
                      a
                      record with the d.id =3D 25, but then you do an MvFIND for an unscoped =
                      id (for
                      example you search id=3D12), it can happen that Empresa will use d.id =
                      =3D 25,
                      because it has a higher priority.=20

                      It is correct that variables passed through the command line are global =
                      by
                      default. Obviously, in the case of your variable "keysearch_first" there =
                      is
                      no ambiguity because there aren't any system vars with that name and for =
                      a
                      database variable the name is too long.

                      However, as soon as you have lots of unscoped variables floating around =
                      in
                      your script, errors happen very quickly. It really doesn't cost much to
                      write but a lot to debug, if you don't use them. Aside from the security
                      aspect, this might even speed up your program (a few microseconds...).


                      About the MvFIND: You should also rtrim(toupper(g.searchterm)) the
                      searchterm in the MvFIND.=20

                      NAME=3D"crckeyword_alias"
                      VALUE=3D"{ RTRIM(toupper(keysearch_first))}"
                      EXACT=3D"RANGE">


                      Indices are case-sensitive. So maybe that's the reason why you don't get =
                      any
                      matches.=20

                      Markus




                      =20

                      -----Original Message-----
                      From: [email protected] [mailto:[email protected]] On =
                      Behalf
                      Of John Brebner
                      Sent: Freitag, 11. M=E4rz 2005 17:21
                      To: MvMarkus
                      Cc: [email protected]
                      Subject: RE: [meu] Problem with index search

                      Hi Markus,

                      Tried that EXACT=3D"RANGE" addition, and now the eof() is reached with =
                      no
                      results for any search term. With regard to the variable scope, aren't =
                      the
                      variables global by default? Then it shouldn't make a difference if I
                      replaced keysearch_first with g.keysearch_first?

                      John


                      At 05:02 PM 3/11/2005 +0100, you wrote:
                      >Hi John
                      >
                      >Two things: Ivo mentioned this already, but you should definitely scope =

                      >your variables. This is probably NOT the reason for the error, but=20
                      >that's a main cause for nasty bugs and even security issues in a =
                      program.
                      >
                      >Then try to change your MvFIND in this way:
                      >
                      ><MvFIND
                      > NAME=3D"crckeyword_alias"
                      > VALUE=3D"{keysearch_first}"
                      > EXACT=3D"RANGE">
                      >
                      >By default, MvFIND tries to use the exact term, if "RANGE" is not=20
                      >explicitly indicated. If you have it in there, then it'll work as Ivo
                      explained.
                      >
                      >Although the RANGE-flag exists for quite a long time already, it hasn't =

                      >made it yet into the html-docs, but is still buried in one of the =
                      readme's.
                      >
                      >Hope that helps,
                      >
                      >Markus
                      >
                      >
                      >-----Original Message-----
                      >From: [email protected] [mailto:[email protected]] On=20
                      >Behalf Of John Brebner
                      >Sent: Freitag, 11. M=E4rz 2005 16:45
                      >To: Ivo Truxa
                      >Cc: [email protected]
                      >Subject: RE: [meu] Problem with index search
                      >
                      >Hi Ivo,
                      >
                      >Here's the code used for creating the index. All keywords are in upper=20
                      >case and left trimmed. The imagekey.dbf has only three fields,=20
                      >"keyword", "negnumber" and "digital".
                      >
                      ><MvOPEN
                      > NAME=3D"crckeyword_alias"
                      > DATABASE=3D"imagekey.dbf"
                      > TYPE=3D"xbase3"
                      > >
                      ><MvMAKEINDEX
                      > NAME=3D"crckeyword_alias"
                      > INDEXFILE=3D"crckeyword.mvx"
                      > EXPR=3D"{keyword $ negnumber}"
                      > >
                      ><MvCLOSE NAME=3D"crckeyword_alias">
                      >
                      >The search term, first_keyword is input by the user.
                      >
                      ><MvOPEN
                      > NAME=3D"crckeyword_alias"
                      > DATABASE=3D"imagekey.dbf"
                      > TYPE=3D"xbase3"
                      > INDEXES=3D"crckeyword.mvx"
                      > >
                      >
                      ><MvOPEN
                      > NAME=3D"crcneg_alias"
                      > DATABASE=3D"crcneg.dbf"
                      > TYPE=3D"xbase3"
                      > INDEXES=3D"crcneg_number.mvx"
                      > >
                      ><MvASSIGN NAME=3D"keysearch_first" =20
                      >VALUE=3D"{RTRIM(toupper(first_keyword))}">
                      ><MvFIND
                      > NAME=3D"crckeyword_alias"
                      > VALUE=3D"{keysearch_first}"
                      > >
                      >
                      >(rest snipped...)
                      >
                      >When you enter a search term that is NOT in the keyword index, then
                      >eof() is reached. If "network" is input, and the term is in the keyword =

                      >database, then no results are shown. Term "networks", "network lab"
                      >etc. produce results.
                      >
                      >I hope this gives you enough information!
                      >
                      >John
                      >
                      >
                      >At 04:04 PM 3/11/2005 +0100, you wrote:
                      > >Hi John,
                      > >
                      > >If both the search expression and the index key value begin with
                      'network'
                      > >(case sensitive), then certainly yes, it should work. If it is this=20
                      > >case, then your index expression is probably incorrectly written, or=20
                      > >the index file invalid, or you do not use the same database alias=20
                      > >consistently (if at all). Posting your code creating the index could=20
                      > >tell
                      >more.
                      > >
                      > >But if the index key contains the word somewhere in the middle of=20
                      > >multiple keywords, then it will not work, of course.
                      > >
                      > >PS: please hit the "reply all" button when replying to posts from the =

                      > >list, otherwise the answer goes only privately to the author.
                      > >
                      > >Ivo
                      > >http://mivo.truxoft.com
                      > >
                      > >
                      > >-----Original Message-----
                      > >From: John Brebner [mailto:[email protected]]
                      > >
                      > >
                      > >Hi Ivo,
                      > >
                      > >Yes, I'm using MvFIND without EXACT set...should that not return all=20
                      > >records containing "network", including "network", "network lab" etc?
                      > >
                      > >John
                      > >
                      > >At 02:45 PM 3/11/2005 +0100, you wrote:
                      > >
                      > >
                      > > Well, you did not write what code you use, but I assume you=20
                      > >try it with
                      > > MvFIND. That won't work so simply for the task you mention.
                      > >MvFIND only
                      > > finds records where the search expression, depending on the=20
                      > >switch EXACT,
                      > > matches the index key either exactly (from the first to the=20
                      > >last character),
                      > > or partially (but starting with the first character). This=20
                      > >is apparently not
                      > > the case in the example you posted.
                      > >
                      > > There are different possibilities to solve the dilemma: the=20
                      > >simplest, but
                      > > also the slowest, is using the MvFILTER command instead.
                      > >Another possibility
                      > > is combining MvFIND and MvFILTER with a special technique=20
                      > >described on my
                      > > website in the resource section. Yet another approach is=20
                      > >accessing the
                      > > problem from the other side - creating an indexed database=20
                      > >of keywords, with
                      > > second field containing ID of all records containing it -=20
                      > >you can then use
                      > > MvFIND to get the list of records immediately.
                      > >
                      > > Ivo Truxa
                      > >
                      > > | http://miva.truxoft.com <http://miva.truxoft.com/>
                      > > | Advanced Miva Merchant modules
                      > >
                      > >
                      > >
                      > > -----Original Message-----
                      > > From: [email protected]=20
                      > >[mailto:[email protected]]
                      > >On Behalf
                      > > Of John Brebner
                      > > Sent: Friday, March 11, 2005 2:26 PM
                      > > To: [email protected]
                      > > Subject: [meu] Problem with index search
                      > >
                      > > Hi everyone,
                      > >
                      > > I'm having a problem with generating results from a simple=20
                      > >index, and I
                      > > can't for the life of me figure out what the problem might =
                      be.
                      > >I'm sure
                      > > it's something obvious, but before I tear out what's left of =

                      > >my hair,
                      > > perhaps someone could offer some suggestions.
                      > >
                      > > I photographic negative database of about 140,000 entries,=20
                      > >with a series of
                      > > keyword fields. These are added to a keyword database of=20
                      > >400,000 entries
                      > > and is indexed on keyword $ image_number. The search program =

                      > >looks for a
                      > > submitted keyword, then outputs the appropriate thumbnail
                      images.
                      > >
                      > > The problem is this: When searching for a term , ie:
                      > >"network", that term
                      > > will NOT be found if their are other keywords such as=20
                      > >"networks", "network
                      > > lab", etc. The longer terms are always found. The search=20
                      > >terms are RTRIMed
                      > > and appear to be passed to the search program properly.
                      > >
                      > > Ideas?
                      > >
                      > > Thanks
                      > >
                      > > John
                      > >
                      > >
                      > >
                      > > John A. Brebner
                      > >
                      > >
                      > >
                      > >John A. Brebner
                      > >
                      > >Researching the Brebner/Bremner families in the North-east of=20
                      > >Scotland and around the world at www.brebner.com
                      <<A HREF ="http://www.brebner.com/ ">http://www.brebner.com/ </A>>


                      Comment


                        #12
                        Problem with index search



                        --=====================_30823391==_.ALT
                        Content-Type: text/plain; charset="us-ascii"; format=flowed

                        Hi Markus,

                        As a last resort I'll change all the variables in the program to
                        g.whatever...but I'd prefer to find a solution to the indexing problem first.

                        The keyword index in all upper case; the search terms are RTRImmed and
                        UCASed. I tried adding your solution of:

                        NAME="crckeyword_alias"
                        VALUE="{ RTRIM(toupper(keysearch_first))}"
                        EXACT="RANGE">

                        and the search finds nothing and goes to eof() with any keyword. So I've
                        gone back to

                        <MvASSIGN NAME="keysearch_first" VALUE="{RTRIM(toupper(first_keyword))}">

                        <MvFIND
                        NAME="crckeyword_alias"
                        VALUE="{keysearch_first}"
                        >

                        which works in this infuriating manner!

                        I simply don't understand why it works fine for any unique keyword in the
                        index, but not with non-unique keyword roots?

                        So that "access door" or "access tunnel" show all entries, but "access"
                        shows none?

                        Still tearing my hair out...but appreciate all the help!

                        John



                        At 05:44 PM 3/11/2005 +0100, you wrote:
                        >Hi John,
                        >
                        >About scoping: It is more a question of "best practice". An unscoped
                        >variable does not immediately default to global. Instead, the system will
                        >first look if it can find a value for var among the system variables
                        >(s.var), then in the database d.var, then a global g.var and finally as a
                        >local l.var.
                        >
                        >So if you have a database column called ID and the record cursor sits at a
                        >record with the d.id = 25, but then you do an MvFIND for an unscoped id (for
                        >example you search id=12), it can happen that Empresa will use d.id = 25,
                        >because it has a higher priority.
                        >
                        >It is correct that variables passed through the command line are global by
                        >default. Obviously, in the case of your variable "keysearch_first" there is
                        >no ambiguity because there aren't any system vars with that name and for a
                        >database variable the name is too long.
                        >
                        >However, as soon as you have lots of unscoped variables floating around in
                        >your script, errors happen very quickly. It really doesn't cost much to
                        >write but a lot to debug, if you don't use them. Aside from the security
                        >aspect, this might even speed up your program (a few microseconds...).
                        >
                        >
                        >About the MvFIND: You should also rtrim(toupper(g.searchterm)) the
                        >searchterm in the MvFIND.
                        >
                        >NAME="crckeyword_alias"
                        > VALUE="{ RTRIM(toupper(keysearch_first))}"
                        > EXACT="RANGE">
                        >
                        >
                        >Indices are case-sensitive. So maybe that's the reason why you don't get any
                        >matches.
                        >
                        >Markus
                        >
                        >
                        >

                        (rest of thread snipped for brevity)


                        --=====================_30823391==_.ALT--

                        Comment


                          #13
                          Problem with index search



                          The easy answer is to just change your index. When
                          combining multiple fields to create an index it's (in my
                          experience) best practice to use a delimiter between them.
                          So change the MvIndex to:
                          > <MvMAKEINDEX
                          > NAME="crckeyword_alias"
                          > INDEXFILE="crckeyword.mvx"
                          > EXPR="{rtrim( ltrim( tolower( l.keyword ) ) ) $
                          '|' $ l.negnumber}"

                          Now make the search <MvFIND NAME="crckeyword_alias"
                          VALUE="{rtrim( ltrim( tolower( l.keyword ) ) ) $ '|' $
                          l.negnumber}">

                          This way you have values like "network|12345", and cannot
                          be confused with "networks|9876" or "network thingies|666".

                          /Scott McC
                          --- John Brebner <[email protected]> wrote:

                          > Hi Ivo,
                          >
                          > Here's the code used for creating the index. All keywords
                          > are in upper case
                          > and left trimmed. The imagekey.dbf has only three fields,
                          > "keyword",
                          > "negnumber" and "digital".
                          >
                          > <MvOPEN
                          > NAME="crckeyword_alias"
                          > DATABASE="imagekey.dbf"
                          > TYPE="xbase3"
                          > >
                          > <MvMAKEINDEX
                          > NAME="crckeyword_alias"
                          > INDEXFILE="crckeyword.mvx"
                          > EXPR="{keyword $ negnumber}"
                          > >
                          > <MvCLOSE NAME="crckeyword_alias">
                          >
                          > The search term, first_keyword is input by the user.
                          >
                          > <MvOPEN
                          > NAME="crckeyword_alias"
                          > DATABASE="imagekey.dbf"
                          > TYPE="xbase3"
                          > INDEXES="crckeyword.mvx"
                          > >
                          >
                          > <MvOPEN
                          > NAME="crcneg_alias"
                          > DATABASE="crcneg.dbf"
                          > TYPE="xbase3"
                          > INDEXES="crcneg_number.mvx"
                          > >
                          > <MvASSIGN NAME="keysearch_first"
                          > VALUE="{RTRIM(toupper(first_keyword))}">
                          > <MvFIND
                          > NAME="crckeyword_alias"
                          > VALUE="{keysearch_first}"
                          > >
                          >
                          > (rest snipped...)
                          >
                          > When you enter a search term that is NOT in the keyword
                          > index, then eof()
                          > is reached. If "network" is input, and the term is in the
                          > keyword database,
                          > then no results are shown. Term "networks", "network lab"
                          > etc. produce results.
                          >
                          > I hope this gives you enough information!
                          >
                          > John
                          >
                          >
                          > At 04:04 PM 3/11/2005 +0100, you wrote:
                          > >Hi John,
                          > >
                          > >If both the search expression and the index key value
                          > begin with 'network'
                          > >(case sensitive), then certainly yes, it should work. If
                          > it is this case,
                          > >then your index expression is probably incorrectly
                          > written, or the index
                          > >file invalid, or you do not use the same database alias
                          > consistently (if at
                          > >all). Posting your code creating the index could tell
                          > more.
                          > >
                          > >But if the index key contains the word somewhere in the
                          > middle of multiple
                          > >keywords, then it will not work, of course.
                          > >
                          > >PS: please hit the "reply all" button when replying to
                          > posts from the list,
                          > >otherwise the answer goes only privately to the author.
                          > >
                          > >Ivo
                          > >http://mivo.truxoft.com
                          > >
                          > >
                          > >-----Original Message-----
                          > >From: John Brebner [mailto:[email protected]]
                          > >
                          > >
                          > >Hi Ivo,
                          > >
                          > >Yes, I'm using MvFIND without EXACT set...should that
                          > not return all records
                          > >containing "network", including "network", "network lab"
                          > etc?
                          > >
                          > >John
                          > >
                          > >At 02:45 PM 3/11/2005 +0100, you wrote:
                          > >
                          > >
                          > > Well, you did not write what code you use, but
                          > I assume you try it
                          > >with
                          > > MvFIND. That won't work so simply for the task
                          > you mention. MvFIND
                          > >only
                          > > finds records where the search expression,
                          > depending on the switch
                          > >EXACT,
                          > > matches the index key either exactly (from the
                          > first to the last
                          > >character),
                          > > or partially (but starting with the first
                          > character). This is
                          > >apparently not
                          > > the case in the example you posted.
                          > >
                          > > There are different possibilities to solve the
                          > dilemma: the
                          > >simplest, but
                          > > also the slowest, is using the MvFILTER command
                          > instead. Another
                          > >possibility
                          > > is combining MvFIND and MvFILTER with a special
                          > technique described
                          > >on my
                          > > website in the resource section. Yet another
                          > approach is accessing
                          > >the
                          > > problem from the other side - creating an
                          > indexed database of
                          > >keywords, with
                          > > second field containing ID of all records
                          > containing it - you can
                          > >then use
                          > > MvFIND to get the list of records immediately.
                          > >
                          > > Ivo Truxa
                          > >
                          > > | http://miva.truxoft.com
                          > <http://miva.truxoft.com/>
                          > > | Advanced Miva Merchant modules
                          > >
                          > >
                          > >
                          > > -----Original Message-----
                          > > From: [email protected]
                          > [mailto:[email protected]]
                          > >On Behalf
                          > > Of John Brebner
                          > > Sent: Friday, March 11, 2005 2:26 PM
                          > > To: [email protected]
                          > > Subject: [meu] Problem with index search
                          > >
                          > > Hi everyone,
                          > >
                          > > I'm having a problem with generating results
                          > from a simple index,
                          > >and I
                          > > can't for the life of me figure out what the
                          > problem might be. I'm
                          > >sure
                          > > it's something obvious, but before I tear out
                          > what's left of my
                          > >hair,
                          > > perhaps someone could offer some suggestions.
                          > >
                          > > I photographic negative database of about
                          > 140,000 entries, with a
                          > >series of
                          > > keyword fields. These are added to a keyword
                          > database of 400,000
                          > >entries
                          > > and is indexed on keyword $ image_number. The
                          > search program looks
                          > >for a
                          > > submitted keyword, then outputs the appropriate
                          > thumbnail images.
                          > >
                          > > The problem is this: When searching for a term
                          > , ie: "network", that
                          > >term
                          > > will NOT be found if their are other keywords
                          > such as "networks",
                          > >"network
                          > > lab", etc. The longer terms are always found.
                          > The search terms are
                          > >RTRIMed
                          > > and appear to be passed to the search program
                          > properly.
                          > >
                          > > Ideas?
                          > >
                          > > Thanks
                          > >
                          > > John
                          > >
                          > >
                          > >
                          > > John A. Brebner
                          > >
                          > >
                          > >
                          >
                          === message truncated ===




                          __________________________________
                          Do you Yahoo!?
                          Yahoo! Small Business - Try our new resources site!
                          http://smallbusiness.yahoo.com/resources/

                          Comment


                            #14
                            Problem with index search



                            Hi John,
                            =20
                            I'm afraid that your problem cannot be easily guessed without further
                            information. I'm pretty sure that it'll be some kind of combination of =
                            the
                            issues that Ivo and I already mentioned.=20

                            One additional aspect comes to mind: Index expressions are stored in the
                            database, and are -just like unscoped variables- vulnerable to =
                            ambiguities.
                            That means, when you create your index (Makeindex) and you do not =
                            indicate
                            the database alias in the expression, Empresa will not reliably know =
                            what
                            value to use. This result can be quite disastrous: When you then add a
                            record, the index value can refer to either crckeyword_alias OR=20
                            crcneg_alias OR - whatever table is the primary in that very moment.=20

                            It can even refer to a system,global or local variable, if the variable =
                            is
                            not scoped.=20

                            So in the moment, when you added "network" to your table, the table =
                            might
                            not have be the primary db, so actually during the add the value used =
                            for
                            the index could have been a value from a (probably inexisting) column of =
                            a
                            different table! Then, when you added the next record (network card, now
                            your crckeyword_alias may be indeed the primary DB, and now the correct
                            value is used to write the index. That's why the first one isn't there,
                            while the second is.

                            You can test it easily:
                            Open the crckeyword_alias with its index
                            Loop through it until the end and display every record. See if the value
                            "network" appears.=20

                            If it doesn't then the value didn't make it into the index.=20




                            Another way is to open the db with the index, get d.totrec (number of
                            physical records) and then do an MvGO =3D bottom and display d.recno. If =
                            this
                            number is different than d.totrec, then some values may not be indexed =
                            at
                            all.

                            Try to correct this by recreating the index with the correct and =
                            complete
                            index expression, using the alias, scope and field name.=20

                            Good luck,

                            Markus






                            ________________________________

                            From: John Brebner [mailto:[email protected]]=20
                            Sent: Freitag, 11. M=E4rz 2005 19:02
                            To: MvMarkus
                            Cc: [email protected]
                            Subject: RE: [meu] Problem with index search


                            Hi Markus,

                            As a last resort I'll change all the variables in the program to
                            g.whatever...but I'd prefer to find a solution to the indexing problem
                            first.

                            The keyword index in all upper case; the search terms are RTRImmed and
                            UCASed. I tried adding your solution of:

                            NAME=3D"crckeyword_alias"
                            VALUE=3D"{ RTRIM(toupper(keysearch_first))}"
                            EXACT=3D"RANGE">

                            and the search finds nothing and goes to eof() with any keyword. So =
                            I've
                            gone back to

                            <MvASSIGN NAME=3D"keysearch_first" =
                            VALUE=3D"{RTRIM(toupper(first_keyword))}">

                            <MvFIND
                            NAME=3D"crckeyword_alias"
                            VALUE=3D"{keysearch_first}"=20
                            >

                            which works in this infuriating manner!

                            I simply don't understand why it works fine for any unique keyword in =
                            the
                            index, but not with non-unique keyword roots?

                            So that "access door" or "access tunnel" show all entries, but "access"
                            shows none?

                            Still tearing my hair out...but appreciate all the help!

                            John



                            At 05:44 PM 3/11/2005 +0100, you wrote:


                            Hi John,
                            =09
                            About scoping: It is more a question of "best practice". An unscoped
                            variable does not immediately default to global. Instead, the system
                            will
                            first look if it can find a value for var among the system variables
                            (s.var), then in the database d.var, then a global g.var and finally
                            as a
                            local l.var.
                            =09
                            So if you have a database column called ID and the record cursor
                            sits at a
                            record with the d.id =3D 25, but then you do an MvFIND for an unscoped
                            id (for
                            example you search id=3D12), it can happen that Empresa will use d.id
                            =3D 25,
                            because it has a higher priority.=20
                            =09
                            It is correct that variables passed through the command line are
                            global by
                            default. Obviously, in the case of your variable "keysearch_first"
                            there is
                            no ambiguity because there aren't any system vars with that name and
                            for a
                            database variable the name is too long.
                            =09
                            However, as soon as you have lots of unscoped variables floating
                            around in
                            your script, errors happen very quickly. It really doesn't cost much
                            to
                            write but a lot to debug, if you don't use them. Aside from the
                            security
                            aspect, this might even speed up your program (a few
                            microseconds...).
                            =09
                            =09
                            About the MvFIND: You should also rtrim(toupper(g.searchterm)) the
                            searchterm in the MvFIND.=20
                            =09
                            NAME=3D"crckeyword_alias"
                            VALUE=3D"{ RTRIM(toupper(keysearch_first))}"
                            EXACT=3D"RANGE">
                            =09
                            =09
                            Indices are case-sensitive. So maybe that's the reason why you don't
                            get any
                            matches.=20
                            =09
                            Markus
                            =09
                            =09
                            =09
                            =09
                            =20
                            =09
                            -----Original Message-----
                            From: [email protected] [mailto:[email protected]]
                            On Behalf
                            Of John Brebner
                            Sent: Freitag, 11. M=E4rz 2005 17:21
                            To: MvMarkus
                            Cc: [email protected]
                            Subject: RE: [meu] Problem with index search
                            =09
                            Hi Markus,
                            =09
                            Tried that EXACT=3D"RANGE" addition, and now the eof() is reached with
                            no
                            results for any search term. With regard to the variable scope,
                            aren't the
                            variables global by default? Then it shouldn't make a difference if
                            I
                            replaced keysearch_first with g.keysearch_first?
                            =09
                            John
                            =09
                            =09
                            At 05:02 PM 3/11/2005 +0100, you wrote:
                            >Hi John
                            >
                            >Two things: Ivo mentioned this already, but you should definitely
                            scope=20
                            >your variables. This is probably NOT the reason for the error, but=20
                            >that's a main cause for nasty bugs and even security issues in a
                            program.
                            >
                            >Then try to change your MvFIND in this way:
                            >
                            ><MvFIND
                            > NAME=3D"crckeyword_alias"
                            > VALUE=3D"{keysearch_first}"
                            > EXACT=3D"RANGE">
                            >
                            >By default, MvFIND tries to use the exact term, if "RANGE" is not=20
                            >explicitly indicated. If you have it in there, then it'll work as
                            Ivo
                            explained.
                            >
                            >Although the RANGE-flag exists for quite a long time already, it
                            hasn't=20
                            >made it yet into the html-docs, but is still buried in one of the
                            readme's.
                            >
                            >Hope that helps,
                            >
                            >Markus
                            >
                            >
                            >-----Original Message-----
                            >From: [email protected] [mailto:[email protected]]
                            On=20
                            >Behalf Of John Brebner
                            >Sent: Freitag, 11. M=E4rz 2005 16:45
                            >To: Ivo Truxa
                            >Cc: [email protected]
                            >Subject: RE: [meu] Problem with index search
                            >
                            >Hi Ivo,
                            >
                            >Here's the code used for creating the index. All keywords are in
                            upper=20
                            >case and left trimmed. The imagekey.dbf has only three fields,=20
                            >"keyword", "negnumber" and "digital".
                            >
                            ><MvOPEN
                            > NAME=3D"crckeyword_alias"
                            > DATABASE=3D"imagekey.dbf"
                            > TYPE=3D"xbase3"
                            > >
                            ><MvMAKEINDEX
                            > NAME=3D"crckeyword_alias"
                            > INDEXFILE=3D"crckeyword.mvx"
                            > EXPR=3D"{keyword $ negnumber}"
                            > >
                            ><MvCLOSE NAME=3D"crckeyword_alias">
                            >
                            >The search term, first_keyword is input by the user.
                            >
                            ><MvOPEN
                            > NAME=3D"crckeyword_alias"
                            > DATABASE=3D"imagekey.dbf"
                            > TYPE=3D"xbase3"
                            > INDEXES=3D"crckeyword.mvx"
                            > >
                            >
                            ><MvOPEN
                            > NAME=3D"crcneg_alias"
                            > DATABASE=3D"crcneg.dbf"
                            > TYPE=3D"xbase3"
                            > INDEXES=3D"crcneg_number.mvx"
                            > >
                            ><MvASSIGN NAME=3D"keysearch_first" =20
                            >VALUE=3D"{RTRIM(toupper(first_keyword))}">
                            ><MvFIND
                            > NAME=3D"crckeyword_alias"
                            > VALUE=3D"{keysearch_first}"
                            > >
                            >
                            >(rest snipped...)
                            >
                            >When you enter a search term that is NOT in the keyword index, then

                            >eof() is reached. If "network" is input, and the term is in the
                            keyword=20
                            >database, then no results are shown. Term "networks", "network lab"

                            >etc. produce results.
                            >
                            >I hope this gives you enough information!
                            >
                            >John
                            >
                            >
                            >At 04:04 PM 3/11/2005 +0100, you wrote:
                            > >Hi John,
                            > >
                            > >If both the search expression and the index key value begin with
                            'network'
                            > >(case sensitive), then certainly yes, it should work. If it is
                            this=20
                            > >case, then your index expression is probably incorrectly written,
                            or=20
                            > >the index file invalid, or you do not use the same database alias

                            > >consistently (if at all). Posting your code creating the index
                            could=20
                            > >tell
                            >more.
                            > >
                            > >But if the index key contains the word somewhere in the middle of

                            > >multiple keywords, then it will not work, of course.
                            > >
                            > >PS: please hit the "reply all" button when replying to posts from
                            the=20
                            > >list, otherwise the answer goes only privately to the author.
                            > >
                            > >Ivo
                            > >http://mivo.truxoft.com <http://mivo.truxoft.com/>=20
                            > >
                            > >
                            > >-----Original Message-----
                            > >From: John Brebner [mailto:[email protected]]
                            > >
                            > >
                            > >Hi Ivo,
                            > >
                            > >Yes, I'm using MvFIND without EXACT set...should that not return
                            all=20
                            > >records containing "network", including "network", "network lab"
                            etc?
                            > >
                            > >John
                            > >
                            > >At 02:45 PM 3/11/2005 +0100, you wrote:
                            > >
                            > >
                            > > Well, you did not write what code you use, but I assume
                            you=20
                            > >try it with
                            > > MvFIND. That won't work so simply for the task you
                            mention.
                            > >MvFIND only
                            > > finds records where the search expression, depending on
                            the=20
                            > >switch EXACT,
                            > > matches the index key either exactly (from the first to
                            the=20
                            > >last character),
                            > > or partially (but starting with the first character).
                            This=20
                            > >is apparently not
                            > > the case in the example you posted.
                            > >
                            > > There are different possibilities to solve the dilemma:
                            the=20
                            > >simplest, but
                            > > also the slowest, is using the MvFILTER command instead.
                            > >Another possibility
                            > > is combining MvFIND and MvFILTER with a special
                            technique=20
                            > >described on my
                            > > website in the resource section. Yet another approach is

                            > >accessing the
                            > > problem from the other side - creating an indexed
                            database=20
                            > >of keywords, with
                            > > second field containing ID of all records containing it
                            -=20
                            > >you can then use
                            > > MvFIND to get the list of records immediately.
                            > >
                            > > Ivo Truxa
                            > >
                            > > | http://miva.truxoft.com <http://miva.truxoft.com/>
                            <http://miva.truxoft.com/>
                            > > | Advanced Miva Merchant modules
                            > >
                            > >
                            > >
                            > > -----Original Message-----
                            > > From: [email protected]=20
                            > >[mailto:[email protected]]
                            > >On Behalf
                            > > Of John Brebner
                            > > Sent: Friday, March 11, 2005 2:26 PM
                            > > To: [email protected]
                            > > Subject: [meu] Problem with index search
                            > >
                            > > Hi everyone,
                            > >
                            > > I'm having a problem with generating results from a
                            simple=20
                            > >index, and I
                            > > can't for the life of me figure out what the problem
                            might be.
                            > >I'm sure
                            > > it's something obvious, but before I tear out what's
                            left of=20
                            > >my hair,
                            > > perhaps someone could offer some suggestions.
                            > >
                            > > I photographic negative database of about 140,000
                            entries,=20
                            > >with a series of
                            > > keyword fields. These are added to a keyword database of

                            > >400,000 entries
                            > > and is indexed on keyword $ image_number. The search
                            program=20
                            > >looks for a
                            > > submitted keyword, then outputs the appropriate
                            thumbnail
                            images.
                            > >
                            > > The problem is this: When searching for a term , ie:
                            > >"network", that term
                            > > will NOT be found if their are other keywords such as=20
                            > >"networks", "network
                            > > lab", etc. The longer terms are always found. The search

                            > >terms are RTRIMed
                            > > and appear to be passed to the search program properly.
                            > >
                            > > Ideas?
                            > >
                            > > Thanks
                            > >
                            > > John
                            > >
                            > >
                            > >
                            > > John A. Brebner
                            > >
                            > >
                            > >
                            > >John A. Brebner
                            > >
                            > >Researching the Brebner/Bremner families in the North-east of=20
                            > >Scotland and around the world at www.brebner.com
                            <<A HREF ="http://www.brebner.com/>=20">http://www.brebner.com/>=20</A>
                            <<A HREF ="http://www.brebner.com/ ">http://www.brebner.com/ </A>>
                            > >
                            > >
                            > >

                            Comment


                              #15
                              Problem with index search



                              Hi Markus (and Ivo)

                              Thanks to both of you for your suggestions. I'm going to take a break from=
                              =20
                              this for the weekend and have another go on Monday, hopefully refreshed.

                              In reference to your comments about adding records, and creating the index,=
                              =20
                              I should say that this is only done once a week. All the new weekly records=
                              =20
                              are added within dBase daily, copied to the mivamia/data directory weekly,=
                              =20
                              then the MvMakeindex goes to work on the updated database that overwrites=20
                              the previous weeks data. So both the negative database and the keyword=20
                              database are indexed sequentially, and in fact, since only one ever opened=
                              =20
                              at any time, the question of which is primary is moot. I've checked, all=20
                              new keywords appear after each update.

                              But I'll give you're method a whirl on Monday!

                              Many thanks,

                              John


                              At 08:10 PM 3/11/2005 +0100, you wrote:
                              >Hi John,
                              >
                              >I'm afraid that your problem cannot be easily guessed without further
                              >information. I'm pretty sure that it'll be some kind of combination of the
                              >issues that Ivo and I already mentioned.
                              >
                              >One additional aspect comes to mind: Index expressions are stored in the
                              >database, and are -just like unscoped variables- vulnerable to ambiguities.
                              >That means, when you create your index (Makeindex) and you do not indicate
                              >the database alias in the expression, Empresa will not reliably know what
                              >value to use. This result can be quite disastrous: When you then add a
                              >record, the index value can refer to either crckeyword_alias OR
                              >crcneg_alias OR - whatever table is the primary in that very moment.
                              >
                              >It can even refer to a system,global or local variable, if the variable is
                              >not scoped.
                              >
                              >So in the moment, when you added "network" to your table, the table might
                              >not have be the primary db, so actually during the add the value used for
                              >the index could have been a value from a (probably inexisting) column of a
                              >different table! Then, when you added the next record (network card, now
                              >your crckeyword_alias may be indeed the primary DB, and now the correct
                              >value is used to write the index. That's why the first one isn't there,
                              >while the second is.
                              >
                              >You can test it easily:
                              >Open the crckeyword_alias with its index
                              >Loop through it until the end and display every record. See if the value
                              >"network" appears.
                              >
                              >If it doesn't then the value didn't make it into the index.
                              >
                              >
                              >
                              >
                              >Another way is to open the db with the index, get d.totrec (number of
                              >physical records) and then do an MvGO =3D bottom and display d.recno. If=
                              this
                              >number is different than d.totrec, then some values may not be indexed at
                              >all.
                              >
                              >Try to correct this by recreating the index with the correct and complete
                              >index expression, using the alias, scope and field name.
                              >
                              >Good luck,
                              >
                              >Markus
                              >
                              >
                              >
                              >
                              >
                              >
                              >________________________________
                              >
                              >From: John Brebner [mailto:[email protected]]
                              >Sent: Freitag, 11. M=E4rz 2005 19:02
                              >To: MvMarkus
                              >Cc: [email protected]
                              >Subject: RE: [meu] Problem with index search
                              >
                              >
                              >Hi Markus,
                              >
                              >As a last resort I'll change all the variables in the program to
                              >g.whatever...but I'd prefer to find a solution to the indexing problem
                              >first.
                              >
                              >The keyword index in all upper case; the search terms are RTRImmed and
                              >UCASed. I tried adding your solution of:
                              >
                              >NAME=3D"crckeyword_alias"
                              > VALUE=3D"{ RTRIM(toupper(keysearch_first))}"
                              > EXACT=3D"RANGE">
                              >
                              >and the search finds nothing and goes to eof() with any keyword. So I've
                              >gone back to
                              >
                              ><MvASSIGN NAME=3D"keysearch_first" =
                              VALUE=3D"{RTRIM(toupper(first_keyword))}">
                              >
                              ><MvFIND
                              > NAME=3D"crckeyword_alias"
                              > VALUE=3D"{keysearch_first}"
                              > >
                              >
                              >which works in this infuriating manner!
                              >
                              >I simply don't understand why it works fine for any unique keyword in the
                              >index, but not with non-unique keyword roots?
                              >
                              >So that "access door" or "access tunnel" show all entries, but "access"
                              >shows none?
                              >
                              >Still tearing my hair out...but appreciate all the help!
                              >
                              >John
                              >
                              >
                              >
                              >At 05:44 PM 3/11/2005 +0100, you wrote:
                              >
                              >
                              > Hi John,
                              >
                              > About scoping: It is more a question of "best practice". An=
                              unscoped
                              > variable does not immediately default to global. Instead, the=
                              system
                              >will
                              > first look if it can find a value for var among the system=
                              variables
                              > (s.var), then in the database d.var, then a global g.var and=
                              finally
                              >as a
                              > local l.var.
                              >
                              > So if you have a database column called ID and the record cursor
                              >sits at a
                              > record with the d.id =3D 25, but then you do an MvFIND for an=
                              unscoped
                              >id (for
                              > example you search id=3D12), it can happen that Empresa will use=
                              d.id
                              >=3D 25,
                              > because it has a higher priority.
                              >
                              > It is correct that variables passed through the command line are
                              >global by
                              > default. Obviously, in the case of your variable "keysearch_first"
                              >there is
                              > no ambiguity because there aren't any system vars with that name=
                              and
                              >for a
                              > database variable the name is too long.
                              >
                              > However, as soon as you have lots of unscoped variables floating
                              >around in
                              > your script, errors happen very quickly. It really doesn't cost=
                              much
                              >to
                              > write but a lot to debug, if you don't use them. Aside from the
                              >security
                              > aspect, this might even speed up your program (a few
                              >microseconds...).
                              >
                              >
                              > About the MvFIND: You should also rtrim(toupper(g.searchterm)) the
                              > searchterm in the MvFIND.
                              >
                              > NAME=3D"crckeyword_alias"
                              > VALUE=3D"{ RTRIM(toupper(keysearch_first))}"
                              > EXACT=3D"RANGE">
                              >
                              >
                              > Indices are case-sensitive. So maybe that's the reason why you=
                              don't
                              >get any
                              > matches.
                              >
                              > Markus
                              >
                              >
                              >
                              >
                              >
                              >
                              > -----Original Message-----
                              > From: [email protected] [mailto:[email protected]]
                              >On Behalf
                              > Of John Brebner
                              > Sent: Freitag, 11. M=E4rz 2005 17:21
                              > To: MvMarkus
                              > Cc: [email protected]
                              > Subject: RE: [meu] Problem with index search
                              >
                              > Hi Markus,
                              >
                              > Tried that EXACT=3D"RANGE" addition, and now the eof() is reached=
                              with
                              >no
                              > results for any search term. With regard to the variable scope,
                              >aren't the
                              > variables global by default? Then it shouldn't make a difference=
                              if
                              >I
                              > replaced keysearch_first with g.keysearch_first?
                              >
                              > John
                              >
                              >
                              > At 05:02 PM 3/11/2005 +0100, you wrote:
                              > >Hi John
                              > >
                              > >Two things: Ivo mentioned this already, but you should definitely
                              >scope
                              > >your variables. This is probably NOT the reason for the error,=
                              but
                              > >that's a main cause for nasty bugs and even security issues in a
                              >program.
                              > >
                              > >Then try to change your MvFIND in this way:
                              > >
                              > ><MvFIND
                              > > NAME=3D"crckeyword_alias"
                              > > VALUE=3D"{keysearch_first}"
                              > > EXACT=3D"RANGE">
                              > >
                              > >By default, MvFIND tries to use the exact term, if "RANGE" is not
                              > >explicitly indicated. If you have it in there, then it'll work as
                              >Ivo
                              > explained.
                              > >
                              > >Although the RANGE-flag exists for quite a long time already, it
                              >hasn't
                              > >made it yet into the html-docs, but is still buried in one of the
                              >readme's.
                              > >
                              > >Hope that helps,
                              > >
                              > >Markus
                              > >
                              > >
                              > >-----Original Message-----
                              > >From: [email protected]=
                              [mailto:[email protected]]
                              >On
                              > >Behalf Of John Brebner
                              > >Sent: Freitag, 11. M=E4rz 2005 16:45
                              > >To: Ivo Truxa
                              > >Cc: [email protected]
                              > >Subject: RE: [meu] Problem with index search
                              > >
                              > >Hi Ivo,
                              > >
                              > >Here's the code used for creating the index. All keywords are in
                              >upper
                              > >case and left trimmed. The imagekey.dbf has only three fields,
                              > >"keyword", "negnumber" and "digital".
                              > >
                              > ><MvOPEN
                              > > NAME=3D"crckeyword_alias"
                              > > DATABASE=3D"imagekey.dbf"
                              > > TYPE=3D"xbase3"
                              > > >
                              > ><MvMAKEINDEX
                              > > NAME=3D"crckeyword_alias"
                              > > INDEXFILE=3D"crckeyword.mvx"
                              > > EXPR=3D"{keyword $ negnumber}"
                              > > >
                              > ><MvCLOSE NAME=3D"crckeyword_alias">
                              > >
                              > >The search term, first_keyword is input by the user.
                              > >
                              > ><MvOPEN
                              > > NAME=3D"crckeyword_alias"
                              > > DATABASE=3D"imagekey.dbf"
                              > > TYPE=3D"xbase3"
                              > > INDEXES=3D"crckeyword.mvx"
                              > > >
                              > >
                              > ><MvOPEN
                              > > NAME=3D"crcneg_alias"
                              > > DATABASE=3D"crcneg.dbf"
                              > > TYPE=3D"xbase3"
                              > > INDEXES=3D"crcneg_number.mvx"
                              > > >
                              > ><MvASSIGN NAME=3D"keysearch_first"
                              > >VALUE=3D"{RTRIM(toupper(first_keyword))}">
                              > ><MvFIND
                              > > NAME=3D"crckeyword_alias"
                              > > VALUE=3D"{keysearch_first}"
                              > > >
                              > >
                              > >(rest snipped...)
                              > >
                              > >When you enter a search term that is NOT in the keyword index,=
                              then
                              >
                              > >eof() is reached. If "network" is input, and the term is in the
                              >keyword
                              > >database, then no results are shown. Term "networks", "network=
                              lab"
                              >
                              > >etc. produce results.
                              > >
                              > >I hope this gives you enough information!
                              > >
                              > >John
                              > >
                              > >
                              > >At 04:04 PM 3/11/2005 +0100, you wrote:
                              > > >Hi John,
                              > > >
                              > > >If both the search expression and the index key value begin=
                              with
                              > 'network'
                              > > >(case sensitive), then certainly yes, it should work. If it is
                              >this
                              > > >case, then your index expression is probably incorrectly=
                              written,
                              >or
                              > > >the index file invalid, or you do not use the same database=
                              alias
                              >
                              > > >consistently (if at all). Posting your code creating the index
                              >could
                              > > >tell
                              > >more.
                              > > >
                              > > >But if the index key contains the word somewhere in the middle=
                              of
                              >
                              > > >multiple keywords, then it will not work, of course.
                              > > >
                              > > >PS: please hit the "reply all" button when replying to posts=
                              from
                              >the
                              > > >list, otherwise the answer goes only privately to the author.
                              > > >
                              > > >Ivo
                              > > >http://mivo.truxoft.com <http://mivo.truxoft.com/>
                              > > >
                              > > >
                              > > >-----Original Message-----
                              > > >From: John Brebner [mailto:[email protected]]
                              > > >
                              > > >
                              > > >Hi Ivo,
                              > > >
                              > > >Yes, I'm using MvFIND without EXACT set...should that not=
                              return
                              >all
                              > > >records containing "network", including "network", "network=
                              lab"
                              >etc?
                              > > >
                              > > >John
                              > > >
                              > > >At 02:45 PM 3/11/2005 +0100, you wrote:
                              > > >
                              > > >
                              > > > Well, you did not write what code you use, but I=
                              assume
                              >you
                              > > >try it with
                              > > > MvFIND. That won't work so simply for the task you
                              >mention.
                              > > >MvFIND only
                              > > > finds records where the search expression, depending=
                              on
                              >the
                              > > >switch EXACT,
                              > > > matches the index key either exactly (from the first=
                              to
                              >the
                              > > >last character),
                              > > > or partially (but starting with the first character).
                              >This
                              > > >is apparently not
                              > > > the case in the example you posted.
                              > > >
                              > > > There are different possibilities to solve the=
                              dilemma:
                              >the
                              > > >simplest, but
                              > > > also the slowest, is using the MvFILTER command=
                              instead.
                              > > >Another possibility
                              > > > is combining MvFIND and MvFILTER with a special
                              >technique
                              > > >described on my
                              > > > website in the resource section. Yet another approach=
                              is
                              >
                              > > >accessing the
                              > > > problem from the other side - creating an indexed
                              >database
                              > > >of keywords, with
                              > > > second field containing ID of all records containing=
                              it
                              >-
                              > > >you can then use
                              > > > MvFIND to get the list of records immediately.
                              > > >
                              > > > Ivo Truxa
                              > > >
                              > > > | http://miva.truxoft.com <http://miva.truxoft.com/>
                              ><http://miva.truxoft.com/>
                              > > > | Advanced Miva Merchant modules
                              > > >
                              > > >
                              > > >
                              > > > -----Original Message-----
                              > > > From: [email protected]
                              > > >[mailto:[email protected]]
                              > > >On Behalf
                              > > > Of John Brebner
                              > > > Sent: Friday, March 11, 2005 2:26 PM
                              > > > To: [email protected]
                              > > > Subject: [meu] Problem with index search
                              > > >
                              > > > Hi everyone,
                              > > >
                              > > > I'm having a problem with generating results from a
                              >simple
                              > > >index, and I
                              > > > can't for the life of me figure out what the problem
                              >might be.
                              > > >I'm sure
                              > > > it's something obvious, but before I tear out what's
                              >left of
                              > > >my hair,
                              > > > perhaps someone could offer some suggestions.
                              > > >
                              > > > I photographic negative database of about 140,000
                              >entries,
                              > > >with a series of
                              > > > keyword fields. These are added to a keyword database=
                              of
                              >
                              > > >400,000 entries
                              > > > and is indexed on keyword $ image_number. The search
                              >program
                              > > >looks for a
                              > > > submitted keyword, then outputs the appropriate
                              >thumbnail
                              > images.
                              > > >
                              > > > The problem is this: When searching for a term , ie:
                              > > >"network", that term
                              > > > will NOT be found if their are other keywords such as
                              > > >"networks", "network
                              > > > lab", etc. The longer terms are always found. The=
                              search
                              >
                              > > >terms are RTRIMed
                              > > > and appear to be passed to the search program=
                              properly.
                              > > >
                              > > > Ideas?
                              > > >
                              > > > Thanks
                              > > >
                              > > > John
                              > > >
                              > > >
                              > > >
                              > > > John A. Brebner
                              > > >
                              > > >
                              > > >
                              > > >John A. Brebner
                              > > >
                              > > >Researching the Brebner/Bremner families in the North-east of
                              > > >Scotland and around the world at www.brebner.com
                              ><<A HREF ="http://www.brebner.com/>">http://www.brebner.com/></A>
                              > <<A HREF ="http://www.brebner.com/ ">http://www.brebner.com/ </A>>
                              > > >
                              > > >
                              > > >

                              Comment

                              Working...
                              X