Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 112 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.6801. Min Range: -5.6801

Max Act: 5.6801. Min Act: -0.1700

Data Index: 2844128 (Open Web Text)

Max Activating Token Index: 35

Text #1

Max Range: 5.6801. Min Range: -5.6801

Max Act: 5.6729. Min Act: -0.1700

Data Index: 5372698 (Open Web Text)

Max Activating Token Index: 533

Text #2

Max Range: 5.6801. Min Range: -5.6801

Max Act: 5.6715. Min Act: -0.1700

Data Index: 5630581 (Open Web Text)

Max Activating Token Index: 221

Text #3

Max Range: 5.6801. Min Range: -5.6801

Max Act: 5.6616. Min Act: -0.1700

Data Index: 2311750 (Open Web Text)

Max Activating Token Index: 184

Text #4

Max Range: 5.6801. Min Range: -5.6801

Max Act: 5.6442. Min Act: -0.1700

Data Index: 1166097 (Open Web Text)

Max Activating Token Index: 425

Text #5

Max Range: 5.6801. Min Range: -5.6801

Max Act: 5.6092. Min Act: -0.1700

Data Index: 8135573 (Open Web Text)

Max Activating Token Index: 217

Text #6

Max Range: 5.6801. Min Range: -5.6801

Max Act: 5.6004. Min Act: -0.1700

Data Index: 2711255 (Open Web Text)

Max Activating Token Index: 194

Text #7

Max Range: 5.6801. Min Range: -5.6801

Max Act: 5.6134. Min Act: -0.1700

Data Index: 3553187 (Open Web Text)

Max Activating Token Index: 131

Text #8

Max Range: 5.6801. Min Range: -5.6801

Max Act: 5.5931. Min Act: -0.1700

Data Index: 1487470 (Open Web Text)

Max Activating Token Index: 312

Text #9

Max Range: 5.6801. Min Range: -5.6801

Max Act: 5.5601. Min Act: -0.1700

Data Index: 5998482 (Open Web Text)

Max Activating Token Index: 356

Text #10

Max Range: 5.6801. Min Range: -5.6801

Max Act: 5.5748. Min Act: -0.1700

Data Index: 5944628 (Open Web Text)

Max Activating Token Index: 218

Text #11

Max Range: 5.6801. Min Range: -5.6801

Max Act: 5.5827. Min Act: -0.1700

Data Index: 6049198 (Open Web Text)

Max Activating Token Index: 477

Text #12

Max Range: 5.6801. Min Range: -5.6801

Max Act: 5.5730. Min Act: -0.1700

Data Index: 6160926 (Open Web Text)

Max Activating Token Index: 520

Text #13

Max Range: 5.6801. Min Range: -5.6801

Max Act: 5.5861. Min Act: -0.1700

Data Index: 7225882 (Open Web Text)

Max Activating Token Index: 140

Text #14

Max Range: 5.6801. Min Range: -5.6801

Max Act: 5.5878. Min Act: -0.1700

Data Index: 7381534 (Open Web Text)

Max Activating Token Index: 48

Text #15

Max Range: 5.6801. Min Range: -5.6801

Max Act: 5.5765. Min Act: -0.1700

Data Index: 3086766 (Open Web Text)

Max Activating Token Index: 228

Text #16

Max Range: 5.6801. Min Range: -5.6801

Max Act: 5.5865. Min Act: -0.1700

Data Index: 7466885 (Open Web Text)

Max Activating Token Index: 438

Text #17

Max Range: 5.6801. Min Range: -5.6801

Max Act: 5.5767. Min Act: -0.1700

Data Index: 7754890 (Open Web Text)

Max Activating Token Index: 690

Text #18

Max Range: 5.6801. Min Range: -5.6801

Max Act: 5.5745. Min Act: -0.1700

Data Index: 7938266 (Open Web Text)

Max Activating Token Index: 280

Text #19

Max Range: 5.6801. Min Range: -5.6801

Max Act: 5.5879. Min Act: -0.1700

Data Index: 5076154 (Open Web Text)

Max Activating Token Index: 393