Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2993 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0187. Min Range: -4.0187

Max Act: 4.0187. Min Act: -0.1700

Data Index: 6838748 (Open Web Text)

Max Activating Token Index: 284

Text #1

Max Range: 4.0187. Min Range: -4.0187

Max Act: 3.9867. Min Act: -0.1700

Data Index: 5115853 (Open Web Text)

Max Activating Token Index: 731

Text #2

Max Range: 4.0187. Min Range: -4.0187

Max Act: 3.9162. Min Act: -0.1700

Data Index: 3499748 (Open Web Text)

Max Activating Token Index: 653

Text #3

Max Range: 4.0187. Min Range: -4.0187

Max Act: 3.8313. Min Act: -0.1700

Data Index: 5787411 (Open Web Text)

Max Activating Token Index: 765

Text #4

Max Range: 4.0187. Min Range: -4.0187

Max Act: 3.8436. Min Act: -0.1700

Data Index: 8121121 (Open Web Text)

Max Activating Token Index: 403

Text #5

Max Range: 4.0187. Min Range: -4.0187

Max Act: 3.8113. Min Act: -0.1700

Data Index: 8742432 (Open Web Text)

Max Activating Token Index: 365

Text #6

Max Range: 4.0187. Min Range: -4.0187

Max Act: 3.8130. Min Act: -0.1700

Data Index: 6182730 (Open Web Text)

Max Activating Token Index: 176

Text #7

Max Range: 4.0187. Min Range: -4.0187

Max Act: 3.8097. Min Act: -0.1700

Data Index: 2854436 (Open Web Text)

Max Activating Token Index: 262

Text #8

Max Range: 4.0187. Min Range: -4.0187

Max Act: 3.7953. Min Act: -0.1700

Data Index: 4767316 (Open Web Text)

Max Activating Token Index: 229

Text #9

Max Range: 4.0187. Min Range: -4.0187

Max Act: 3.8006. Min Act: -0.1700

Data Index: 4335065 (Open Web Text)

Max Activating Token Index: 642

Text #10

Max Range: 4.0187. Min Range: -4.0187

Max Act: 3.8298. Min Act: -0.1700

Data Index: 5343972 (Open Web Text)

Max Activating Token Index: 928

Text #11

Max Range: 4.0187. Min Range: -4.0187

Max Act: 3.8062. Min Act: -0.1700

Data Index: 2804444 (Open Web Text)

Max Activating Token Index: 220

Text #12

Max Range: 4.0187. Min Range: -4.0187

Max Act: 3.8021. Min Act: -0.1700

Data Index: 3819414 (Open Web Text)

Max Activating Token Index: 122

Text #13

Max Range: 4.0187. Min Range: -4.0187

Max Act: 3.7982. Min Act: -0.1700

Data Index: 3135613 (Open Web Text)

Max Activating Token Index: 473

Text #14

Max Range: 4.0187. Min Range: -4.0187

Max Act: 3.8024. Min Act: -0.1700

Data Index: 7377630 (Open Web Text)

Max Activating Token Index: 216

Text #15

Max Range: 4.0187. Min Range: -4.0187

Max Act: 3.7773. Min Act: -0.1700

Data Index: 2007353 (Open Web Text)

Max Activating Token Index: 13

Text #16

Max Range: 4.0187. Min Range: -4.0187

Max Act: 3.7828. Min Act: -0.1700

Data Index: 5715849 (Open Web Text)

Max Activating Token Index: 106

Text #17

Max Range: 4.0187. Min Range: -4.0187

Max Act: 3.7605. Min Act: -0.1700

Data Index: 8561798 (Open Web Text)

Max Activating Token Index: 656

Text #18

Max Range: 4.0187. Min Range: -4.0187

Max Act: 3.7535. Min Act: -0.1700

Data Index: 5976602 (Open Web Text)

Max Activating Token Index: 558

Text #19

Max Range: 4.0187. Min Range: -4.0187

Max Act: 3.7676. Min Act: -0.1700

Data Index: 8557527 (Open Web Text)

Max Activating Token Index: 167