Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1178 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 4.3159. Min Range: -4.3159

Max Act: 4.3159. Min Act: -0.1700

Data Index: 1624082 (Python Code)

Max Activating Token Index: 562

Text #1

Max Range: 4.3159. Min Range: -4.3159

Max Act: 4.0275. Min Act: -0.1700

Data Index: 1614326 (Python Code)

Max Activating Token Index: 252

Text #2

Max Range: 4.3159. Min Range: -4.3159

Max Act: 4.0204. Min Act: -0.1700

Data Index: 1560963 (Python Code)

Max Activating Token Index: 504

Text #3

Max Range: 4.3159. Min Range: -4.3159

Max Act: 3.9757. Min Act: -0.1700

Data Index: 1585283 (Python Code)

Max Activating Token Index: 727

Text #4

Max Range: 4.3159. Min Range: -4.3159

Max Act: 3.9352. Min Act: -0.1700

Data Index: 1461056 (Python Code)

Max Activating Token Index: 358

Text #5

Max Range: 4.3159. Min Range: -4.3159

Max Act: 3.9293. Min Act: -0.1700

Data Index: 1578785 (Python Code)

Max Activating Token Index: 97

Text #6

Max Range: 4.3159. Min Range: -4.3159

Max Act: 3.9546. Min Act: -0.1700

Data Index: 1481957 (Python Code)

Max Activating Token Index: 458

Text #7

Max Range: 4.3159. Min Range: -4.3159

Max Act: 3.9475. Min Act: -0.1700

Data Index: 1573791 (Python Code)

Max Activating Token Index: 842

Text #8

Max Range: 4.3159. Min Range: -4.3159

Max Act: 3.9272. Min Act: -0.1700

Data Index: 1395229 (Python Code)

Max Activating Token Index: 765

Text #9

Max Range: 4.3159. Min Range: -4.3159

Max Act: 3.9198. Min Act: -0.1700

Data Index: 1628067 (Python Code)

Max Activating Token Index: 440

Text #10

Max Range: 4.3159. Min Range: -4.3159

Max Act: 3.8841. Min Act: -0.1700

Data Index: 1149996 (C4 (Web Text))

Max Activating Token Index: 231

Text #11

Max Range: 4.3159. Min Range: -4.3159

Max Act: 3.8692. Min Act: -0.1700

Data Index: 1601271 (Python Code)

Max Activating Token Index: 776

Text #12

Max Range: 4.3159. Min Range: -4.3159

Max Act: 3.8623. Min Act: -0.1700

Data Index: 1527435 (Python Code)

Max Activating Token Index: 582

Text #13

Max Range: 4.3159. Min Range: -4.3159

Max Act: 3.8987. Min Act: -0.1700

Data Index: 1417820 (Python Code)

Max Activating Token Index: 999

Text #14

Max Range: 4.3159. Min Range: -4.3159

Max Act: 3.8689. Min Act: -0.1700

Data Index: 1521824 (Python Code)

Max Activating Token Index: 740

Text #15

Max Range: 4.3159. Min Range: -4.3159

Max Act: 3.8537. Min Act: -0.1700

Data Index: 1587535 (Python Code)

Max Activating Token Index: 299

Text #16

Max Range: 4.3159. Min Range: -4.3159

Max Act: 3.8856. Min Act: -0.1700

Data Index: 1387888 (Python Code)

Max Activating Token Index: 638

Text #17

Max Range: 4.3159. Min Range: -4.3159

Max Act: 3.8577. Min Act: -0.1700

Data Index: 1595839 (Python Code)

Max Activating Token Index: 499

Text #18

Max Range: 4.3159. Min Range: -4.3159

Max Act: 3.8773. Min Act: -0.1700

Data Index: 1448526 (Python Code)

Max Activating Token Index: 492

Text #19

Max Range: 4.3159. Min Range: -4.3159

Max Act: 3.8767. Min Act: -0.1700

Data Index: 1413624 (Python Code)

Max Activating Token Index: 623