Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2169 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.2861. Min Range: -1.2861

Max Act: 1.2861. Min Act: -0.0794

Data Index: 5500368 (Open Web Text)

Max Activating Token Index: 983

Text #1

Max Range: 1.2861. Min Range: -1.2861

Max Act: 1.2768. Min Act: -0.0588

Data Index: 2797236 (Open Web Text)

Max Activating Token Index: 959

Text #2

Max Range: 1.2861. Min Range: -1.2861

Max Act: 1.2456. Min Act: -0.0684

Data Index: 602845 (Open Web Text)

Max Activating Token Index: 1022

Text #3

Max Range: 1.2861. Min Range: -1.2861

Max Act: 1.2269. Min Act: -0.0638

Data Index: 602843 (Open Web Text)

Max Activating Token Index: 960

Text #4

Max Range: 1.2861. Min Range: -1.2861

Max Act: 1.2199. Min Act: -0.0861

Data Index: 6472497 (Open Web Text)

Max Activating Token Index: 970

Text #5

Max Range: 1.2861. Min Range: -1.2861

Max Act: 1.2242. Min Act: -0.0849

Data Index: 4439543 (Open Web Text)

Max Activating Token Index: 961

Text #6

Max Range: 1.2861. Min Range: -1.2861

Max Act: 1.2190. Min Act: -0.1364

Data Index: 6453580 (Open Web Text)

Max Activating Token Index: 1009

Text #7

Max Range: 1.2861. Min Range: -1.2861

Max Act: 1.2198. Min Act: -0.0599

Data Index: 1030527 (Open Web Text)

Max Activating Token Index: 946

Text #8

Max Range: 1.2861. Min Range: -1.2861

Max Act: 1.2169. Min Act: -0.0208

Data Index: 5225070 (Open Web Text)

Max Activating Token Index: 959

Text #9

Max Range: 1.2861. Min Range: -1.2861

Max Act: 1.2174. Min Act: -0.0732

Data Index: 4693892 (Open Web Text)

Max Activating Token Index: 944

Text #10

Max Range: 1.2861. Min Range: -1.2861

Max Act: 1.2149. Min Act: -0.0669

Data Index: 2590055 (Open Web Text)

Max Activating Token Index: 958

Text #11

Max Range: 1.2861. Min Range: -1.2861

Max Act: 1.2096. Min Act: -0.1489

Data Index: 3490177 (Open Web Text)

Max Activating Token Index: 972

Text #12

Max Range: 1.2861. Min Range: -1.2861

Max Act: 1.2043. Min Act: -0.0857

Data Index: 5227602 (Open Web Text)

Max Activating Token Index: 981

Text #13

Max Range: 1.2861. Min Range: -1.2861

Max Act: 1.2037. Min Act: -0.0741

Data Index: 4206938 (Open Web Text)

Max Activating Token Index: 944

Text #14

Max Range: 1.2861. Min Range: -1.2861

Max Act: 1.2035. Min Act: -0.1261

Data Index: 934254 (Open Web Text)

Max Activating Token Index: 945

Text #15

Max Range: 1.2861. Min Range: -1.2861

Max Act: 1.2088. Min Act: -0.0531

Data Index: 3728063 (Open Web Text)

Max Activating Token Index: 895

Text #16

Max Range: 1.2861. Min Range: -1.2861

Max Act: 1.2031. Min Act: -0.0693

Data Index: 3513538 (Open Web Text)

Max Activating Token Index: 423

Text #17

Max Range: 1.2861. Min Range: -1.2861

Max Act: 1.2074. Min Act: -0.0932

Data Index: 602844 (Open Web Text)

Max Activating Token Index: 993

Text #18

Max Range: 1.2861. Min Range: -1.2861

Max Act: 1.2009. Min Act: -0.0892

Data Index: 5163776 (Open Web Text)

Max Activating Token Index: 909

Text #19

Max Range: 1.2861. Min Range: -1.2861

Max Act: 1.2029. Min Act: -0.0631

Data Index: 2175846 (Open Web Text)

Max Activating Token Index: 907