Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 496 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4608. Min Range: -3.4608

Max Act: 3.4608. Min Act: -0.1700

Data Index: 4319473 (Open Web Text)

Max Activating Token Index: 365

Text #1

Max Range: 3.4608. Min Range: -3.4608

Max Act: 3.4229. Min Act: -0.1700

Data Index: 7116009 (Open Web Text)

Max Activating Token Index: 203

Text #2

Max Range: 3.4608. Min Range: -3.4608

Max Act: 3.3636. Min Act: -0.1700

Data Index: 197614 (Open Web Text)

Max Activating Token Index: 483

Text #3

Max Range: 3.4608. Min Range: -3.4608

Max Act: 3.3527. Min Act: -0.1700

Data Index: 6313851 (Open Web Text)

Max Activating Token Index: 351

Text #4

Max Range: 3.4608. Min Range: -3.4608

Max Act: 3.3207. Min Act: -0.1700

Data Index: 3653888 (Open Web Text)

Max Activating Token Index: 525

Text #5

Max Range: 3.4608. Min Range: -3.4608

Max Act: 3.3098. Min Act: -0.1700

Data Index: 8565343 (Open Web Text)

Max Activating Token Index: 730

Text #6

Max Range: 3.4608. Min Range: -3.4608

Max Act: 3.2961. Min Act: -0.1700

Data Index: 8507108 (Open Web Text)

Max Activating Token Index: 527

Text #7

Max Range: 3.4608. Min Range: -3.4608

Max Act: 3.3066. Min Act: -0.1700

Data Index: 1387433 (Open Web Text)

Max Activating Token Index: 638

Text #8

Max Range: 3.4608. Min Range: -3.4608

Max Act: 3.2986. Min Act: -0.1700

Data Index: 7212763 (Open Web Text)

Max Activating Token Index: 122

Text #9

Max Range: 3.4608. Min Range: -3.4608

Max Act: 3.3060. Min Act: -0.1700

Data Index: 5383894 (Open Web Text)

Max Activating Token Index: 885

Text #10

Max Range: 3.4608. Min Range: -3.4608

Max Act: 3.2984. Min Act: -0.1700

Data Index: 3890032 (Open Web Text)

Max Activating Token Index: 389

Text #11

Max Range: 3.4608. Min Range: -3.4608

Max Act: 3.3055. Min Act: -0.1700

Data Index: 8189132 (Open Web Text)

Max Activating Token Index: 1014

Text #12

Max Range: 3.4608. Min Range: -3.4608

Max Act: 3.2991. Min Act: -0.1700

Data Index: 1310689 (Open Web Text)

Max Activating Token Index: 552

Text #13

Max Range: 3.4608. Min Range: -3.4608

Max Act: 3.3090. Min Act: -0.1700

Data Index: 62514 (Open Web Text)

Max Activating Token Index: 452

Text #14

Max Range: 3.4608. Min Range: -3.4608

Max Act: 3.2931. Min Act: -0.1700

Data Index: 4940116 (Open Web Text)

Max Activating Token Index: 827

Text #15

Max Range: 3.4608. Min Range: -3.4608

Max Act: 3.2977. Min Act: -0.1700

Data Index: 4395370 (Open Web Text)

Max Activating Token Index: 208

Text #16

Max Range: 3.4608. Min Range: -3.4608

Max Act: 3.2979. Min Act: -0.1700

Data Index: 8760411 (Open Web Text)

Max Activating Token Index: 790

Text #17

Max Range: 3.4608. Min Range: -3.4608

Max Act: 3.2800. Min Act: -0.1700

Data Index: 6635344 (Open Web Text)

Max Activating Token Index: 602

Text #18

Max Range: 3.4608. Min Range: -3.4608

Max Act: 3.2663. Min Act: -0.1700

Data Index: 737299 (Open Web Text)

Max Activating Token Index: 973

Text #19

Max Range: 3.4608. Min Range: -3.4608

Max Act: 3.2897. Min Act: -0.1700

Data Index: 2073336 (Open Web Text)

Max Activating Token Index: 698