Entity framework 6, fluent mapping of 1 to 0..1 relationship on non-primary-key columns

I'm trying to migrate an objectcontext/edmx system to dbcontext (EF6) with fluent mapping. I have a few instances like the following. Principal table has a relationship to a dependent table where a non-PK column in the dependent table holds the value of a PK column in the principal. On its own this would be a one to many relationship, but there is a unique index on the dependent table FK columns. Using EDMX mapping, this works fine as long as you define the relationship using mapping rather then referential constraints. Below is an executable example - you don't need the database because it doesn't get that far, currently:

Imports System.Data.Entity
Imports System.Data.Entity.ModelConfiguration
Module Module1

  Sub Main()
    Using session As New SaturnEntities
      Dim res = session.BookingLegSet.Select(Function(x) New With {x.Prefix, x.ID, x.AddressDetails.Address}).ToList
    End Using
  End Sub



End Module

Partial Public Class BookingLeg
  Public Property Prefix As String
  Public Property ID As Integer
  Public Property LegIndex As Integer
  Public Overridable Property AddressDetails As BookingLegAddress
End Class

Partial Public Class BookingLegAddress
  Public Property Prefix As String
  Public Property ID As Integer
  Public Property Address As String
  Public Overridable Property BookingLeg As BookingLeg
  Property JobLegPrefix As String
  Property JobLegID As Integer?


End Class

Public Class BookingLegConfig
  Inherits EntityTypeConfiguration(Of BookingLeg)

  Public Sub New()
    ToTable("JobLegs", "dbo")
    HasKey(Function(x) New With {x.Prefix, x.ID})
    HasOptional(Function(x) x.AddressDetails).WithRequired(Function(x) x.BookingLeg).Map(Function(x) x.MapKey("Prefix", "ID"))
  End Sub
End Class

Public Class BookingLegAddressConfig
  Inherits EntityTypeConfiguration(Of BookingLegAddress)

  Public Sub New()
    ToTable("JobAddresses", "dbo")
    HasKey(Function(x) New With {x.Prefix, x.ID})
    HasRequired(Function(x) x.BookingLeg).WithOptional(Function(x) x.AddressDetails).Map(Function(x) x.MapKey("JobLegPrefix", "JobLegID"))
  End Sub
End Class

Partial Public Class SaturnEntities
  Inherits DbContext


  Public Sub New()
    MyBase.New("data source=dbSaturn;initial catalog=Saturn;integrated security=True;MultipleActiveResultSets=True;")
  End Sub

  Protected Overrides Sub OnModelCreating(modelBuilder As DbModelBuilder)
    modelBuilder.Configurations.Add(New BookingLegConfig)
    modelBuilder.Configurations.Add(New BookingLegAddressConfig)
  End Sub

  Public Overridable Property BookingLegAddressSet() As DbSet(Of BookingLegAddress)
  Public Overridable Property BookingLegSet() As DbSet(Of BookingLeg)
End Class

BookingLeg is the principal entity and BookingLegAddress is the dependant. JobLegPrefix and JobLegID in the dependent will be either null or will hold the Prefix and ID values from a BookingLeg record. When you run this you get an error that AddressDetails has been configured with conflicting mapping information. I've tried numerous different ways to map this, but haven't got anywhere - can anyone tell me what I need to do?